Does anyone know of a good way to easily track this type of thing personally?
For example, I just told a co-worker that I was 90% confident about [work trivia]. It turned out (an entire two minutes later) that I was quite wrong about [work trivia]. I now need to be right about the next 9 things that I have 90% confidence about!
Estimating project time requirements and then tracking time spent has been great for me, as it turns out I tend to estimate about 50% too high.
It would be great to work out if I’m habitually over-confident or over-pessimistic with my predictions, so I can adjust to suit.
I haven’t used this more than once or twice, but this is PredictionBook’s jam! https://predictionbook.com/ I’m pretty sure there are others.
Nearly a decade ago, I worked somewhere that really liked FogBugz, and there was some feature that let PMs see each engineer’s prediction multiplier for task length estimations. I have no idea where that’s landed these days.
On Commoncog, there is an entire series about how the superforecasters do it (e.g. military intelligence, stock analysts, etc). This article explains the rules of the game and the Brier score, which is the metric to track forecasting accuracy:
I have more to say about this topic than I have time to say right now but an Android app that works nicely for this is “LW Predictions”. Or just maintain a document or spreadsheet.
Slate Star Codex makes predictions ever year and scores them and it’s amazing. My favorite quote from the last batch:
I forgot to make predictions for 2020 until now [Apr 8], which in retrospect was the best prediction I’ve ever made. I’ll probably come up with some later this month.