Accuracy and Predictions – It’s not a Magic Formula

Nearly every major news site and political blog offers deep analysis of the latest polls with a rolling prediction of who will win the election. But it’s Nate Silver and his blog www.FiveThirtyEight.com that most people rely on for an accurate Election Day prediction.

That’s because he called the Presidential race for 2008 (getting 49 of 50 states accurately) and 2012 (hitting all 50) with surprising accuracy. But it’s not just his track record that makes him the most credible political pollster in America. His predictions are based on hard data taken from numerous polls combined with historical trends to boost his accuracy. This aggregate poll approach has been widely adopted as the industry norm now, yet Nate still seems to have the edge over everyone else.

I believe it’s his relentless focus on the data to substantiate political narratives, as opposed to partisan news outlets (MSNBC, Fox News, etc.) that let the political narrative shape how they cover the polls. It’s not just having access to good data – it’s about following the right analysis. A forensic approach to discern the truth, regardless of where it leads you.

In fact, my firm Nucleus Research takes that same ‘analytics’ approach to our annual technology predictions, focusing on what the data and end-user customers are telling us. We’ve made some very bold predictions over the past 14 years of issuing our forecast for the year ahead. For example:

• We predicted Google Glass would fail when everyone else was calling it a run-away success.

• 10 years ago, we predicted that Skype would become an enterprise-class application and not just a cool way for Scandinavian teenagers to stay connected.

• We said that Microsoft would replace CEO Steve Ballmer several years before he acutally stepped down.

• Nucleus was considered to be overly bullish on the cloud more than a decade ago as other industry analyst firms hedged bets. In fact we were well ahead of the tipping point.

Our predictions have been dead on most of the time. In fact, we are 95 percent accurate over the past 14 years. Meanwhile, we’ve seen other firms make fairly outlandish predications that over time have proven to be pure folly.

Windows Phone was supposed to overtake iPhone by 2015, said one firm. Another – the industry giant so to say – predicted that Windows Vista would be the ‘last version of Windows that exists in its current, monolithic form.” I’ve enjoyed watching their analysts discuss Windows, 7, 8 and the current version Windows 10.

Sadly, these analyst firm colleagues aren’t doing our industry any favors with their ‘shoot-from-the-hip’ narrative-driven predictions that often fall flat. They are more Madame Zelda the psychic than Nate Silver.

And that’s because it comes back to that narrative-driven approach. If you spend most of your time with vendors, their flashy marketing spin will start to take hold. Honestly, I see too many predictions that line up to what vendors need in order to be successful. (Partisan results from publicly traded firms that need to make their numbers?) I’m not saying we shouldn’t listen to vendors, but it’s only one side of the equation. It gives us great insight into capabilities and what to expect in the coming year as far as products.

But it’s the end-user customers who really give the best indication of what trends will take off. And where Nate Silver aggregates data from multiple polls, Nucleus aggregates data from enterprise customers. This is what allows us to have the pulse of what’s really happening and what’s about to emerge as the next trend.

Just as Nate Silver will discount the occasional outlier poll and focus on the data that confirms the real trend, we assess customer feedback to see the common thread. This comes naturally to us since we are close to customers with our ROI case studies, value matrices and tools that help companies justify IT spending. The vendors let us know what to expect from them, but the customers understand where they plan to invest and what they intend to deploy over the next 12 months. Even discussing their pain points gives insight into what’s on the horizon. It also helps us understand and even guide vendors willing to align to customer needs.

Now I don’t want to start a discussion on who’s going to win the election, but my money is on Nate Silver. And I’ve very confident that our tech predictions for next year will be accurate and instructive too. But go ahead and judge for yourself. You can read them at 2017 Predictions.