This is brilliant:
Google’s horrible promotion polices discourage important work and encourage optimizing for the wrong thing.
A good thing to remember when setting up your Bee-goals.
A great example of Goodhart’s law!
5 Likes
This is absolutely true. I have been targeting for a certain number of hours of study per day for many years now. And I agree that at times, I am just unnecessarily spending a lot of time on things that can be done quickly. A good reminder for me. Thanks.
4 Likes
Definitely good to have Goodhart’s Law in mind but I believe Beeminder is pretty robust to it:
https://blog.beeminder.com/unintended/
Also related: “loopholes you can drive an ice cream truck through”. I think the existence of ice-cream-truck loopholes proves that Beeminder can be robust to Goodhart’s Law. Of course not everyone (and I vaguely recall from previous discussions that @zedmango may be one such user) can relate to ice-cream-truck loopholes and if that’s you then Goodhart’s Law may be a particularly apt warning.
2 Likes
So Goodhart’s law is tool-agnostic. It applies to the goals you set and it doesn’t matter what tools you use to help you achieve those goals.
Example: I want to be happy, I think X will bring me happiness, so I make a Beeminder goal to get X. I successfully use Beeminder (or any other tool) to get X, but in fact X didn’t make me happy.
So it’s not really meaningful to say that any tool can be more or less robust to Goodhart’s Law - Goodhart’s Law is a warning about setting the right goals for you and you can’t blame your tools on that.
One possible protection might be to Beemind a regular review of your goals and see if they’re actually getting you what you want - that is, why did you originally think the goal was a good idea, and does it still serve you?
5 Likes
From the Wikipedia article on Goodhart’s Law, it sounds like it applies specifically to statistical measures used for management leverage, which can lead to unintended consequences as those subject to the new measure do everything they can to move the number, without regard to the broader goal the target was supposed to support.
I feel like Beeminder may not be very likely to fall into that trap because:
- I’m the one managing myself, so I know exactly the unintended effects the goal is having with nearly no delay.
- I can quickly take steps to change, replace, or remove a goal that is no longer serving my best interests.
Even so, I have had goals hang around for weeks or even months beyond their useful lives. While they probably didn’t have a large negative effect on my life (otherwise I would have caught them sooner), there is an opportunity cost to leaving an old goal in place when a new goal might be more effective.
[Edit: oops! Sorry for the bump 
]
2 Likes
Hear hear!
PS: No need to apologize for bumping the thread! That perfectly articulated the case I was trying to argue.
1 Like
It’s easy to give Google’s promotion scheme flak but I would argue that at least part of the reason why that’s true is because the promotion scheme is actually well specified and pretty consistently enforced. The “traditional” scheme of your manager going into a smoke-filled room and arguing with his peers on your behalf (or trading you for a pack of cigarettes) can’t really be analyzed because there’s no actual objective criteria.
The Google scheme also has the advantage that your manager can truly be your champion in the process because they can’t affect the outcome but they benefit from the result (i.e. their report being promoted benefits them). So when they say “Damn, I thought you deserved the promotion. Let’s regroup and make a plan for next time.” it can actually be true because they didn’t just backstab you in the promotion process.
My point is just let’s not confuse the framework (a neutral committee of strangers reviewing your work) from the criteria inside that framework (focus on quantitative metrics showing impact to the company).
1 Like
The general principle is broader than management leverage, though.
It’s true that with Beeminder, you’re the one managing yourself, but sometimes you’re the one least aware of how things are affecting your life. A particular activity or schedule or diet may be negatively affecting your mood or energy level, but those around you might realize before you do. I think people often keep goals that aren’t serving them due to inertia.
I’m not sure I understand. The problem with Google’s scheme is not that a committee reviews your work, and it’s not that they focus on quantitative metrics - it’s that the specific metrics they use don’t serve them. They’re encouraging useless work and discouraging genuinely helpful work.
The other problem is that the structure of the review doesn’t allow employees or their advocates to actually explain why the work is valuable.
1 Like