Beware of false precision

dilbert -model

Human beings crave certainty and in the world of product management it translates to “estimates of value”. Product managers have to make tradeoffs regularly on what initiates to work out next. At a certain stage of the company the need for “formal estimates of the value of doing X” will kick in, otherwise, how do you know to work on X or Y? There will be an overwhelming desire to quantify everything to the nth detail before deciding what to do next. We all know the dangers of that – If you torture excel enough you will get the answer you want. Use of a complicated bottom-up model in the early stages of a product’s evolution is a huge warning sign for me.

How to counter this?

The answer is to sweat the details and remember that simple is better than complicated. It always good to build a top-down and bottom-up model. The top-down model is simple, makes relatively few assumptions to arrive at an estimate. The bottom-up model is more involved, it breaks down your high-level assumptions into smaller chunks. Each of these smaller chunks has their own assumptions. You add everything up to get the final estimate.

Here is the trick, most of the times the two models will differ wildly. Sometimes the top-down will be more conservative and the sometimes the bottom-up will be more conservative. The key thing to remember is that the estimate is what it literally means – its an estimate. You want to be directionally correct – not exact. My rule of thumb has always to adjust the simple top-down model to get as close to the conservative (lower) estimate. This effectively gives you a sense of what the value-floor is and you have fewer assumptions to tweak. It is also extremely important to apply this approach consistently to the initiatives/projects you are evaluating i.e use the top-down model in this case for all the projects as a basis for comparison, don’t mix and match!

You are right in asking – why do all this work on the bottom-up model if I’m going to discard it anyway? The effort is not wasted tho – you now have a jump start on figuring out how to tweak and make a new model once your initial simple assumptions have been ironed out post-launch. Also once you know what assumptions you have to make/are making, it forces you to be clear about the mountain of assumption-debt you are taking on. It’s assumptions all the way down!

Leave a Reply