Clive's intermittent chess improvement journal

[clivemeister]

So, what’s the sitch? Well, I’m what they call an “adult improver” in chess, and I feel seriously in need of an improvement boost!

Which is to say, I’ve returned to chess after an absence of many years, and I’m trying to get better again, and I’m frustrated by my lack of progress. It’s not that I’m actually terrible at chess, in some absolute sense: my platform of choice, Lichess, allows you to see exactly where you are in their population of active chess players, and I am almost exactly in the middle right now - on the 51st centile, in fact:

That’s not bad, but it’s not exactly good, either - and “I know” I’m better than this! I know I could be doing better, and - what’s worse - very definitely I know this is all my fault! It’s my own failure to do what I know I should be doing, that leads me to be where I am - right in the middle of the pack. Chess has no element of chance in it, no “oh the cards didn’t go my way”, no “but the other person was lucky”. It’s just you, and the board, and the other randomly-selected (by Lichess) person at about the same level as you. And it’s not even about how smart you are - turns out, chess isn’t even strongly correlated with general intelligence: the correlation with fluid reasoning is 0.24, and with visuospatial ability is only 0.13 (from this meta-analysis - there are many others!).

And that’s really why I’m starting this. Because chess improvement is, in some ways, the perfect petri dish for what many of us use Beeminder for - getting better/fitter/healthier/more-able-to-leap-small-mole-hills-in-a-single-bound. It’s about stopping doing some things (blundering) whilst doing more of others (spotting great tactics). It’s a good test-case, too, because it’s a tremendously well-studied domain: it’s not like I don’t know - in theory - what I should do. It’s just that, in practice, I’m not doing it.

What’s my goal? I dunno, I just want to not suck so much? But “they” tell me I should make my goals more specific and measureable and time-limited all that. So let’s do that: I want to get from the 50th centile to the 75th centile by the end of this year.

Right then, let’s do this thing! And let’s do it together, if you’re interested - I know I’m not the only one trying to do this! Feel free to comment, kvetch, and contribute! Let’s see if we can work out how to actually get better at chess, and how we can use Beeminder to do it!

[philip]

Fellow “adult improver” here. I knew the rules, of course, but had never really played. Chess forums make me laugh with people saying things like “I’m super old to be learning chess, is it even possible now that I’m 23?” And here’s me at more than twice their age, having signed up to Lichess last summer thanks to the Beeminder integration.

Interesting meta-analysis; I’ve got no mental visualisation skill, so I’ve set up a physical board, which makes it much easier for me to solve puzzles. Learning to think in this more logical structured way has been interesting and is probably useful.

Mostly I’ve been doing puzzles (pjh/vlaakith) and puzzle streaks (untracked), with a weekly game against the computer (pjh/vecna). I suspect that I probably am terrible at actual chess, a suspicion which I have thus far resisted testing against a human opponent. My Lichess mixed puzzle rating is hovering around 1700, and my longest puzzle streak is 21.

[clivemeister]

The first thing I think I need to do, given where I’m typing this, is to work out my first set of Beeminder goals. I already have a couple, using our autodata link to Lichess: one is for playing games of chess on Lichess, while the other is around a course I’m doing from Chessable, solving 10 puzzles a day for 50 days (except I’ve given myself weekends off).

Those are both “input” goals, which is a fine start, as we want goals for things that we directly control. Should I also have an “output” goal, measuring what I’m actually trying to achieve? There are some arguments for not having such a goal, for example that failing at the task is maybe enough of an incentive, but we’re trying to go Full Beeminder here, so I’m going to set one up anyway…

…and it turns out there are 296 days left in 2025 at the time of writing this, and that’s almost exactly the number of Elo points by which I need to improve to get to my stated goal of 75th centile. If I hit it, that would get me to the 76.8th centile as of today, around that’s close enough! (Although that may move about a bit as the Lichess population changes, I guess?) That’s a pretty neat coincidence, and makes my new goal easy to set up: chess_elo. ^[1]

I’m going to start with this, play a few games, see what I think should come next.

@philip

I’ve got no mental visualisation skill

I think many people struggle with this concept, me included! There’s a whole arcana around “what model do you have in your head for a chess board”, and the generally-accepted wisdom is that most people (including GMs) do not have anything like a strong enough visualisation capability to hold a “view” of a full chessboard in their head. The few who do seem to have learned chess very young. But it turns out that it doesn’t matter, so long as you can learn to trust whatever your brain actual is doing (which may be words, or something kinetic, or just some blank space that seems to know what’s going on), and just go with that. Practicing this definitely helps - which unfortunately generally does seem to mean solving puzzles just by gazing at the situation on a board, not by moving the physical pieces.

---

1. Technical note: initially I’d set a rate of +1/day, but then I then modified it with the Commitment Dial to a end_date + value_target goal, I will very likely derail at some point. If I left it as a rate + end_date goal, then any respite pause would lower the value at the end date, and I’d be missing my target. By switching to date + value, I’d still be aiming at the same target number, just at a slightly steeper slope. ↩︎

[philip]

@clivemeister

Should I also have an “output” goal, measuring what I’m actually trying to achieve? There are some arguments for not having such a goal

One such argument comes from Stjepan Tomić of Hanging Pawns, in that worse results in the short term may make you a better player over time. Having said that, he too publicly tracks progress toward an annual ELO target.

IIRC this was in a video where he talked about having abandoned the London System, not because it was ineffective but because many of the middle games were similar, so his repertoire and analysis were the worse for it.

One thing that may stymie Beeminding ELO is that progress seems likely to be lumpy and on occasion retrograde. Another is the presumed need to enter the figure manually.

For my own goals that need daily manual input, I have a script that creates a pessimistic data point every morning, which self destructs on data entry but which sorts to the top of my dashboard and iOS app. Hint: include text like this in the comment: “PESSIMISTIC PRESUMPTION #THISWILLSELFDESTRUCT”

[clivemeister]

Lots of the general research on skill improvement talks about intentional, focused, practice as the way to go for pretty much anything. There’s maybe some debate in general about whether it’s most productive to work on improving your best areas, or on your worst areas - but not for me with chess, at least not at this level. It’s clear that for me (and I think for most players at around my level) that I “lose” more games than I “win” - that is to say, a single blunder will be the reason I lose most of my games, and correspondingly, catching my opponent in a blunder will often be the reason I win.

We need some data! Using the Lichess API, you can retrieve a whole bunch of stuff - and, it turns out, if you use the Lichess system to analyse all your games (which I do), this can include the move you made your first blunder.

So here’s a histogram of when I made my first blunder, in games I lost:

And here it is for games I won:

That’s… very surprising! I thought I was going to see a clear pattern where I blundered later in games I won than in games I lost. But I don’t, which is confusing! Still, science is all about showing your failures as well as your successes, so this was a hypothesis that failed its test… so back to the drawing board, I need to think about this some more!

[clivemeister]

Obviously I’ve been thinking about the surprising pattern in blunders, especially in the games I won. Here’s what I found:

• firstly, the definition of blunder that’s used by Lichess is quite intricate: it makes an estimate of your chances of winning the game, based on the current centipawn score of your position. If your move reduces your winning chance significantly enough, then it’s a blunder. This does mean that the blunder can be something like not spotting a great opportunity to take advantage of a poor move by the opponent, rather than something “bad” that you yourself have done.
• secondly, because in games at my level both players tend to blunder more than once, it’s not necessarily the first blunder that matters, so much as the last blunder.

Of course this still means I need to be trying harder to (a) not make blunders, and (b) spot and take advantage of my opponents blunders more reliably.

I need somehow to loop this back into the general guidance for improving in skills: intentional focused practice. How do I do intentional focused practice on not blundering, and on spotting opponent’s blunders? “Puzzles” are the classic way to do chess tactics training, but my problem with blunders seems like it needs quite specific kinds of training. My problem is loss of focus, rather than pure tactical awareness - and puzzles are a bit weird because you know there’s an answer, and you’re looking for one. I definitely need to improve at this tactics awareness, but it’s the focus thing that I need to improve - how do I train to do the right checks on every single move?

Looking at what people do in other fields, one thing I came up with is when self-taught golf players run into a skill barrier, and get themselves a coach. They then find it difficult to re-build their swing so they are doing it the “right” way, rather than the way they taught themselves. They need to break down their old patterns, and learn new ones - which means getting worse for a while!

So I think I need to “rebuild my swing” in chess. To do this, I’m going to try having an actual physical checklist, and play a bunch of games against the computer while using the checklist every move. (You’re not allowed to do this in person vs person play, any more than you’re allowed to use a computer to check your moves for you!)

I’ll play games like this without time limits for a few weeks so I can start without that extra pressure, before maybe adding time constraints back in. I’ll perhaps print a checklist, or somehow force myself to “tick each box” each move, and create a beeminder goal to play at least 1 goal a day in this way. The idea is to train with the actual checklist until it becomes an automatic habit - to rebuild from the ground up!

[clivemeister]

OK, so it’s been a handful of days, and so far I’m pretty much failing to actually improve - indeed, I’ve lost more games than I’ve won since starting all this. This is at least somewhat because my practice method (of rigorously check-listing before playing a move) isn’t carrying over to my real games (i.e. for Elo points versus humans, rather than computers).

This isn’t surprising, perhaps - early days, and all that. Still, it’s demoralizing, and I need to keep reminding myself that this is a marathon, not a sprint, and if it was that easy then everyone would be doing it!

I’m sure there’s a significant element of my brain thinking that the computer/practice games "don’t count’, so the element of adrenaline which is there in human/for-points games isn’t there. Don’t know what to do with this reflection at this stage, but there it is, in all its ugly glory.

[clivemeister]

Well it’s definitely been downhill for a week or so, with a couple of derailments due to too many losses! This feels horrible when it happens, but after a day or two of thinking about giving up chess and taking up macrame instead, I try to use this as fuel to get me back on track. For now, at least, I’m doing a little better - playing some chess, and winning more than loosing.

One thing that has helped - and it’s obvious, so I’m just embarrassed not to have done it sooner - is to set shorter-term goals and plans, as well as the “by the end of the year I will have X” type of goal. Sure, we still want that. But, specifically, to get there, I’ll need to really make a step change in my game: just being a bit better, and loosing a little less to people at my Elo level, isn’t going to cut it.

A digression on Elo: it’s named after a person, Arpad Elo, so just the initial letter is capitalised. He improved upon an earlier system, and popularised it during the 1960s in the US chess leagues. It works in theory for any two-player games, and is used more widely than just chess now. It is calculated in such a way that if you play someone with an Elo rating 100 points higher than yours, you are expected to win 64% of the time. For 200 Elo points higher, that increases to 76%. More generally, the formula is:

\frac{1}{1 + 10^{\frac{D}{400}}}

Which means that since I’m seeking to gain about 300 Elo, then future-me should be able to beat current-me about 85% of the time.

Returning from the digression, this means a whole bunch of improvement between now and then. Stopping blundering quite so much is one thing I definitely need to do, but I also need to be better and spotting active tactics, and better and the strategy part of the game that puts me in a position to have more tactical opportunities. I saw this in one of my recent games, where I still made a couple of blunders, but also took the strategic initiative, and managed to checkmate my opponent in 27 moves. Felt really good!