CHESSDNADo chess improvement apps actually work?
We measure our own players, attack the result, and re-run it every month. Here is what holds up.
Most players who use Chess DNA gain rating, and more of them beat their own past each month. A modest edge, not proof the app caused it.
277 players, median +35
74% gained
same games, up from 50%
see the correction
Every player's rating change
First to last game after joining, in each player's main time class. All 277 players, including the 79 who lost rating.
- 192 gained, 79 lost, 6 level. Range +552 to −254.
- Median +35, same as last month on a cohort two-thirds bigger. The mean (+54) is inflated by outliers, so we lead with the median.
- Time matters. Measured 18+ days: +59, 74% gained (n=99). Under 18 days: +23, 67% (n=178).
Players like you
Filter the same 277 players by starting rating, time control and how long we've measured them.
Each bar is one player in the group, sorted by rating change. Hover for their numbers.
- Lower-rated players gain faster everywhere in chess, app or not. Compare a band with its own baseline, not with other bands.
- More games, more gain: 100+ games +81 (n=51), 50–99 games +45, 20–49 games +18. Also the most confounded cut, which is why the next test holds game count fixed.
- Treat any group under ~30 players as anecdote.
Did they beat their own past?
Each dot is one player: rating change over N games before joining (across) vs the same N games after (up). Above the diagonal = beat their own past.
| Run | Players | Median lift | Beat own past |
|---|---|---|---|
| Jul 13 | 28 | +4.5 | 50.0% |
| Aug 3 | 57 | +15 | 52.6% |
| Sep 7 | 135 | +7 | 56.3% |
| Oct 15 | 206 | +20 | 58.3% |
- 120 of 206 (58%) beat their own previous run of equal length. First run where chance is unlikely (sign test, p ≈ 0.02).
- The share is the stat to watch. It rose four runs in a row. The lift (+20) is noisy and swings run to run.
- Their "before" was normal. Pre-signup pace was 0.40 points per game, in line with what Chess.com and Lichess data imply for their level (roughly 0.3 to 0.5).
- Not proof of cause. Players who join after a bad stretch often bounce back on their own.
Five attacks on the stat
What a skeptic would say, and what survived.
“You mixed time classes.”
True in our first pass. We now measure inside each player's main time class only. The median went up, not down.
“They just play more.”
Per month, gains look 3.6× faster, but that is mostly play volume. We dropped per-month numbers and now compare equal game counts.
“Outliers prop up the average.”
So we lead with the median (+35), not the mean (+54), and chart every player.
“You only measure users who stuck around.”
Correct. That is survivorship bias, and with no control group it can't be argued away. This is a stat about engaged players.
“New accounts are still calibrating.”
Possible for a few big climbers (1073 → 1625 in five days). Unlikely to explain the whole spread.
69% of active players gained rating, and 58% beat their own equal-length past. Both measured per player, with n attached.
"Chess DNA adds X rating." That is causal, and observational data can't show it.
The stat we got wrong
Published Jul 13 and Aug 3, 2026. Retracted Sep 7, 2026.
We used to say users improved ≈1.7× faster per game than before joining. We got it by dividing one group's median by another's. That compares groups, not people. Paired person by person, the effect was zero.
| Run | What we published | Correct, per player | Beat own pace |
|---|---|---|---|
| Jul 13 | 1.61× | −0.21 pts/game | 46% |
| Aug 3 | 1.41× | −0.20 pts/game | 49% |
| Sep 7 | 1.49× | +0.004 pts/game | 50% |
The gap came from regression to the mean. The rating gains above were always computed per player and are unaffected. The fix is the paired test in section 3.
FAQ
Do chess improvement apps actually work?
For engaged users, measurably and modestly, yes. Across 277 Chess DNA players with 20 or more post-signup games, 69% gained rating with a median of +35; among the 99 we have measured for 18 days or longer, 74% gained with a median of +59. Over the same number of games, 58% beat their own immediately preceding run, up from 50% in our first run, with a median per-user lift of +20 rating points. No app can claim causality from observational data, because motivated players self-select into every sample like this.
How much rating did Chess DNA users gain?
In the October 15, 2026 run, the median change from first to last post-signup game, inside each player's dominant time class, was +35 across 277 players: 192 gained, 79 lost and 6 finished level. The spread ran from +552 to −254 over a median 11-day window. Split by how long we have been measuring, players with an 18-day-or-longer window gained a median +59 (n=99), and players measured for under 18 days gained +23 (n=178). The mean, +54, is pulled up by outliers, so we lead with the median.
Isn't this just selection bias?
Partly, and we say so in the body. Requiring 20 or more post-signup games self-selects motivated players, and churned users never qualify. That is survivorship bias, and without a randomized control group it can only be disclosed, not dismissed. Everything we could test, we tested: time-class mixing, play frequency, outliers, calibrating accounts, and our own per-game multiple, which we retracted in September 2026 after it failed a per-person check. The claims we publish are the ones still standing after all of that.
Why measure improvement per game instead of per month?
Per-month numbers smuggle in play frequency. Users play more after joining an app, so their monthly gains jump even if each game teaches them nothing new; our own per-month number came out 3.6 times faster and we refused to publish it. Per game is the honest denominator, but only if it is computed per person. Comparing the median of one group against the median of another is not a within-person comparison, which is the mistake we made twice. The version we publish holds the game count fixed and pairs every player against their own preceding stretch.
How was this measured?
Every user with 20 or more games after signup in their dominant time class (n=277 on October 15, 2026), internal accounts excluded, with signup proxied by the first imported game. Rating change is last minus first post-signup rating inside that class only, because mixing blitz and rapid ratings fabricates deltas. The paired comparison takes the 206 players who also had enough earlier games: their post-signup run against their own equally long stretch immediately before it, one difference per player, then the median and the share above zero.
Do the numbers hold up over time?
We re-run the identical query monthly and publish every run. The sample has grown 28, 59, 167, 277. The blended median moved +50, +30, +35, +35, driven by cohort mix, because each run adds fresh signups with short windows. Players measured 18 days or longer sat at +50, +96, +55 and +59. The steadiest series is the share who beat their own prior run of equal length: 50.0%, 52.6%, 56.3%, 58.3%, four runs in one direction. Run 3 also caught a methodology error in our own per-game claim, which we retracted in full.
Method. Every Chess DNA user with ≥20 post-signup games in their main time class (signup = first imported game), n=277 on Oct 15, 2026, internal accounts excluded. Rating change = last minus first post-signup rating in that class. Paired test: the 206 players with enough earlier games, each compared with their own equally long stretch just before joining. Past runs are as-of replays. No causal claim. Series: How Chess Games Are Actually Lost, What Each Elo Level Struggles With, Chess Improvement Plateau.