So I ran into a great example of Goodhart’s law today. Goodhart’s law is the observation that people tend to optimize their behavior for the things they measure, not for their actual goals.
It’s a problem with Beeminder and any other tool for reaching goals - it’s a general problem with setting goals and applies to every tool. It’s not something that can be overcome, except by carefully looking at your goals and your actual behavior, re-examining everything, and making changes as necessary.
So here is the example - I set up a whittle-down goal for measuring the number of emails in my inbox. I am trying to clear out my inbox and move them into storage folders. As a first step in moving the emails to storage folders, I moved them to an “archive” folder. The plan was to move them from the archive folder to specific storage folders.
So I set up another whittle-down goal for the “archive” folder. But as soon as I did that, I ran into a problem. Now every time I moved an email from inbox to archive, I was making negative progress on the archive goal - so as a result, I stopped clearing out my inbox.
When I realized what had happened, I changed the archive goal so it now measures the number of emails in my archive folder PLUS the number of emails in the inbox folder. This way, I am not disincentivized from moving an email from inbox to archive!
@oulfis raised another good example, along with a post describing additional Beeminder goals added to try to solve the problem:
I would say Beeminder avoids the Goodhart-type problem where a manager or politician sets a bad metric and the people assessed by it can’t change it, since with Beeminder you are the one setting your own goals.
But any goal-oriented approach is going to be subject to the Goodhart-type problem where you set a goal that seems to make sense, but you end up unconsciously gaming it to some degree - which is why it’s so helpful to set up a regular review to see if your goals are actually working for you.