Chess DNADo chess improvement apps actually work?
Every training app claims it works. We ran the numbers on our own users — 82% gained rating, median +50 — then spent a day trying to kill the result five different ways before publishing it. Here's what survived, what we refuse to claim, and what happened when we re-ran it three weeks later on twice the sample.
Interactive lesson — The hanging piece: the question that saves games: Play it on the board below, and make the key move yourself. White to move. Run the cheapest check in chess before you touch anything, which black pieces have a defender, and which have none? Rxd5. Nothing guards the knight, so the rook simply takes it. No calculation, just counting. Now look for the recapture. The pawn on b6 covers a5 and c5, not d5, and the king is on the other side of the board. A whole piece, won by asking one question. Ask it every move, and ask it for your own pieces too.
First run, Jul 13, 2026: every user with ≥20 post-signup games, n=28 · deltas within each user's dominant time class only · internal accounts excluded. Re-run Aug 3, 2026 with n=59 — see the run table.
What 28 users' ratings actually did
Rating change from each user's first to last post-signup game, inside their dominant time class. Every user in the sample is on this chart — including all five who lost rating.
This is the whole sample, not a highlight reel. 23 of 28 users gained rating; the spread runs from +349 down to −90, and the middle of the distribution — the number a skeptic should care about — is +50. The biggest gainer (352 → 701 in 35 days) is a low-rated account climbing fast, which is exactly the profile you'd flag as a still-calibrating rating. We flag it too: it's one reason the headline here is a median with a sample size attached, not the mean it inflates.
For context on what those users are actually fixing while they climb, the companion reports cover what each Elo level struggles with and how chess games are actually lost — short version: games are decided by one collapse, and shrinking that collapse is where the rating points live.
Faster than their own past
Rating points gained per game played, before vs after joining — the same users, acting as their own baseline (n=26 with ≥20 games on both sides of signup).
The obvious objection to any before/after stat: maybe these players were improving anyway. So we compared them to themselves. For the 26 users who also had 20+ games before signing up, the pace was 0.45 rating points per game before joining and 0.77 after — about 1.7× their own prior pace. Same person, same time class, different slope.
And here's the claim we killed: measured per month, the same data says users improve 3.6× faster after joining. That number is real, spectacular — and dishonest. Users simply play more after joining an improvement app, so a per-month framing smuggles play frequency in as skill. Only the per-game number survives scrutiny, so it's the only one we publish. If you're comparing tools, track your progress in per-game terms too.
How that pace compares to the rest of chess
Per-game rating pace — our users versus what published data implies is typical at each level. The grey bars are estimates (caveats below); our two numbers are measured directly.
A 1.7× jump over your own past only means something if that past was normal to begin with. It was. Our users' pre-signup pace of 0.45 rating points per game lands right on what the wider data implies for their level. Published medians from Chess.com's SmarterChess analysis put a studying 800-rated player near +239 rating a year, and Lichess's public rating data (hundreds of millions of games) shows beginners around 800–1000 gaining their first ~100 points within a few months. Convert either into a per-game figure for an active player and you land in the 0.3–0.5 range — exactly where our users started before joining.
Two honest caveats, because this is a comparison. First, improvement gets harder as you climb: the same external data has a 1600-rated player gaining only about +35 a year, so the grey bars drop fast with level — and our sample skews toward beginners, whose ceiling for quick gains is naturally higher. Second, turning an annual figure into a per-game one means assuming a game volume — the exact move we refused to make with our own per-month stat. So treat the grey bars as a rough band, not a precise line. What is not an estimate: 0.45 before and 0.77 after are both counted per game, directly, on the same accounts. Against even the fastest-improving external band, the post-signup number sits above it.
Five ways we tried to kill the stat
Before publishing, we attacked the result the way a skeptical reviewer would. The attacks and verdicts, unedited.
“You mixed time classes.”
Blitz and rapid ratings live on different scales; compute a delta across them and you fabricate numbers. True — and our first run did exactly that, producing a median of +36. Re-run within each user's dominant time class only, the median rose to +50. The attack was correct, and fixing it made the stat stronger, not weaker.
“They just play more, and more games means more rating.”
Partially lands. Per month, post-signup improvement looks 3.6× faster — but decomposing it shows most of that is play frequency, not play quality. We demoted the claim to the per-game version, ≈1.7×, and retired the per-month framing permanently.
“Outliers prop up the average.”
True in both directions: a +349 climber stays in the sample, while a −661 collapse was excluded as implausible organic play (five days, bullet — the pattern of account sharing, not chess). Verdict: the mean (+66) is decorative. The median (+50) is the stat.
“You're only measuring the users who stuck around.”
Correct, and undismissable. Requiring 20+ post-signup games self-selects motivated players; users who churned never enter the sample. That's survivorship bias, and without a randomized control group — which we don't have — it can't be argued away. So it goes in the article, not in a footnote: this is a stat about engaged users, not about everyone who signs up.
“Provisional ratings regress — your gainers were just calibrating.”
We haven't separated this effect, and our biggest gainer (352 → 701) pattern-matches a new account still finding its level — see regression toward the mean. It's unlikely to explain the whole distribution, but it's one more reason the headline is a median with n attached, not a hero number.
What we will and won't claim
We will claim: 82% of active users gained rating within their first weeks (median +50, n=28), and active users improved ≈1.7× faster per game than their own pace before joining. Both are descriptive, both survived the grill, and both come with their sample size attached. Both also held on the August re-run: the per-game multiple came back ≈1.6× on a doubled sample.
We won't claim: “Chess DNA adds X rating” — that's causal, and observational data can't support causality no matter how much a marketing page wants it to. And we won't claim “3.6× faster improvement,” because we know exactly which confound produces it. When any training tool shows you an improvement stat, ask which of these five attacks they ran against it — and what denominator the claim is hiding. If you'd rather test it on your own games, Chess DNA analyzes them free and shows you the same per-game curve we used here.
Run 2: we re-ran it three weeks later
A static stat is easy to cherry-pick; a re-run schedule is not. On Aug 3, 2026 we ran the identical query again — same ≥20-game bar, same dominant-time-class rule, same exclusions. The sample more than doubled. Every future run gets a row here, including the ones that dip.
| Run | Users | Median gain | % who gained | Median window | Pace vs own past |
|---|---|---|---|---|---|
| Jul 13, 2026 | 28 | +50 | 82% | 18 days | ≈1.7× per game |
| Aug 3, 2026 — everyone | 59 | +30 | 75% | 10.5 days | ≈1.6× per game |
| Aug 3, 2026 — mature window (≥18 days) | 17 | +96 | 88% | ≥18 days | 0.81 pts/game |
Read the blended row honestly and it looks like a dip: median +30, 75% gained. The reason is boring, and it's not the app: the sample doubled with fresh signups — the median measurement window fell from 18 days to 10.5 — and absolute rating gain scales with time on the clock. Split the cohort by window and the story inverts: users measured over the same horizon as run 1 (≥18 days, n=17) show median +96 with 88% gaining — stronger than the number we originally published. The fresh half sits at +18 so far, at roughly half the per-game pace, which is what "hasn't had time yet" looks like in data.
And the one stat this whole article argued you should watch — per-game pace against your own pre-signup self — barely moved: ≈1.7× in July, ≈1.6× now (median 0.45 points per game before joining, 0.71 after, n=52 matched users). No new accounts needed excluding this run; the same five attacks from the grill apply unchanged, including the selection-bias disclosure. That's the whole point of re-running: the flattering number (+50) moved with cohort mix, and the honest number held.
Users who gained rating — the full sample is charted above, including all five who lost.
Median measurement window. These are first-weeks gains, not a year of grinding.
Median rating points gained per game played after joining, across the 28 users.
A small, honest sample — we'd rather publish a modest real number than a big soft one.
Questions people actually ask
Do chess improvement apps actually work?
For engaged users, measurably and modestly, yes: 82% of our users with 20+ post-signup games gained rating (median +50, n=28), at ≈1.7× their own prior per-game pace. What no app can honestly claim from observational data is causality — motivated players self-select into every such sample.
How much rating did Chess DNA users gain?
Median +50, first to last post-signup game, within each user's dominant time class. 23 of 28 gained; the spread ran +349 to −90 over a median 18-day window. The mean (+66) is inflated by one outlier, so we lead with the median.
Isn't this just selection bias?
Partly, and we say so in the article body: requiring 20+ games self-selects motivated users, and churned users never qualify. That's survivorship bias — undismissable without a control group. Everything we could test (class mixing, play frequency, outliers), we tested; this one we disclose.
Why per game instead of per month?
Per-month framing smuggles in play frequency: users play more after joining, so monthly gains jump even if each game teaches nothing. Our per-month number (3.6×) looked spectacular and we refused to publish it. Per game — 1.7× — is the honest denominator.
How was this measured?
Every user with ≥20 games after their first import (n=28), internal accounts excluded plus one implausible account (−661 in five days). Delta = last − first rating within the dominant time class; pace comparison uses the 26 users with ≥20 pre-signup games too.
Do the numbers hold up over time?
We re-ran the identical analysis on Aug 3, 2026, three weeks after publishing. The sample grew from 28 to 59 users; the blended median dipped to +30 (75% gained) because fresh signups with short windows now dominate — but users measured over the same ≥18-day horizon as the original run showed median +96 with 88% gaining, and the per-game pace multiple held at ≈1.6× versus each user's own pre-signup pace. Every re-run gets published in the run table above, including the ones that dip.
Want the same measurement on your own chess? Chess DNA analyzes your games, finds the mistakes that actually cost you rating, and tracks your per-game pace — the number this article just argued is the only one worth watching.
Method. Sample: every Chess DNA user with ≥20 games played after signup (proxied by their first imported game), n=28. Internal accounts excluded, plus one account whose 661-point collapse in five days indicated account sharing rather than organic play. Ratings come from the user's own game records; deltas are last − first post-signup rating computed within each user's dominant time class only — mixing blitz and rapid ratings fabricates deltas (our mixed-class run gave +36; the class-clean run gives +50). The pace comparison uses the 26 users who also had ≥20 pre-signup games: 0.45 points per game before vs 0.77 after. No causal claim is made; see the grill section for the attacks we ran. Part of a series with How Chess Games Are Actually Lost and What Each Elo Level Struggles With.