Tech Predictions Accuracy Challenge: How Often Does your Analyst Firm Get It Right?

Have you ever heard of Hans Hut? No? He predicted the world would end on May 28, 1528. Then there is that group of English astronomers that predicted a London flood on February 1, 1524 to start the apocalypse. I’m not sure anyone remembers their names. But we all remember their contemporary, Nostradamus.

The thing about predictions is that they have to be accurate to have any value beyond pure entertainment. And while history is littered with failed prophets, we continue to see so-called experts swing for the rafters with brash projections for the year ahead.

Take IDC, for example, which predicted in 2011 that Windows phone would eclipse iPhone in 2015. With just over a month to go as of this writing, that is some major ground for Microsoft to cover. Sadly, IDC doubled down on the prediction in 2012, saying that Windows would overtake iPhone in 2016 instead. And as recently as May of this year, IDC predicted that Windows phone would steal market share from iPhone and Android.

Speaking of smart phones, Gartner said in 2009 that Symbian would be the top mobile operating system (OS) in 2012. You know the Nokia OS – that company that Microsoft bought only to kill off earlier this year. And after originally predicting iPhone would be in fourth place, they later revised it back to fifth place behind Blackberry. In fact, both Symbian and Blackberry sank to the bottom in 2012.

Of course it’s easy to highlight a failed prediction, but let’s be honest. With Gartner, IDC and Forrester, there are so many. And I’ve yet to see a track record from any of them. Predictions and the ability to extrapolate on future trends is a primary competency for industry analyst firms. Clients rely on this information to make informed decisions. Shouldn’t it be accurate?

I would be a hypocrite to point at industry peers for not revealing their accuracy rate without doing the same for Nucleus Research. Over the past 12 years, we’ve made 120 predictions with an accuracy rate of 94.2 percent. That’s a number that I’m very proud of and also a metric I run my own business on.

And since nearly every technology on the market will invite a product comparison, I think it is important for all of the industry analyst firms to track their prediction success and transparently report their track record. This is a powerful way to understand who gets it right and who has a pulse on the future. For that reason, I issue the predictions accuracy challenge to Gartner and all other industry analyst firms.

Speaking of pulse, we attribute our high success rate on our close and frequent contact with end-user customers. Of course we speak regularly with vendors to understand their products, vision and future road maps. But that’s only half of the equation – what’s in the pipeline, if you will. It’s only by spending time with customers that we can understand their requirements and which technologies will deliver the most value for them. That customer focus is what helps us make bold predictions that have come true 94.2 percent of the time to date.

It’s neither magic nor a crystal ball. It’s a deep analysis of customer requirements, technology possibilities, and what vendors can actually deliver. So the next time you talk to an analyst firm, ask them about their predictions track record. Give them the accuracy challenge.