Some of My Biases Are Doing a Job, and Removing Them Made Things Worse

The framing around cognitive bias is corrective: here is a systematic error, here is how to reduce it. I spent eighteen months reducing one of mine and I now think the framing is incomplete, because a systematic error can also be doing something useful, and removing it removes both.

The bias was optimism about new projects. I reliably underestimated effort and overestimated the chance of a good outcome, which is well documented and which I could demonstrate in my own records. So I built the correction described in a previous piece: a multiplier on time estimates, and a written pre-mortem before committing to anything.

It worked. My estimates got much better. It also, over the following year, cut the number of things I started by roughly half, and — this is the part I did not anticipate — the projects I did start were not noticeably better than the ones I would have started anyway. The correction did not filter well; it filtered indiscriminately, because a realistic assessment of any ambitious project is discouraging, including the ones that would have gone well.

What I think was happening is that the optimism was not primarily a forecasting error. It was a starting mechanism. Accurate foresight about a difficult undertaking produces the correct answer that it will be hard and might fail, and that answer is not motivating, and things that require motivation do not get started by correct answers.

I do not think this means the bias was good. Two of the projects I abandoned in that period were ones I would not have started under the corrected process, and abandoning them cost me months. The correction genuinely saved me that.

What I have settled on is separating the two functions, which sounds obvious and took me a while. The optimistic estimate is allowed at the point of deciding whether something is worth wanting. The corrected estimate is applied at the point of committing resources — a date, a budget, a promise to someone. Both exist, in writing, and the gap between them is itself informative: a project where the optimistic and realistic versions are close is a different kind of thing from one where they differ by a factor of three.

The broader point I would make is that the debiasing literature is largely about accuracy, and a lot of what our reasoning is doing is not aimed at accuracy. It is aimed at getting a person to act, to commit, to keep going through the part where the evidence looks bad. Those are functions, and a systematically inaccurate belief can serve them.

I am not arguing for keeping comfortable illusions, which is the obvious bad reading. I am arguing that before removing a systematic error it is worth asking what it was load-bearing for, because if the answer is “starting things” then removing it without a replacement leaves you accurate and stationary. My replacement is crude — a rule that I start one thing per quarter regardless of how the realistic estimate looks — and it is worse than the optimism was at its job, and it is the best I have.