Is Big Data becoming a Dirty Word?

If you’re 40+, you may remember 15-20 years ago when data warehousing was the THING. Every company that could afford to build one was either in the process, or thinking about it. According to the pundits at the time, the data warehouse (DW) would revolutionize business. It would give you quicker, deeper, more actionable insights and allow you to have foresight as well, so that you could make predictions about your future and better decisions on how to leverage that future to your advantage.

Unfortunately, even though there were many DW success stories (think Wal Mart and Amazon) there were also some major horror stories. Many companies that tried to build enterprise DWs fell into a quagmire of bungled requirements, dirty data, and technology overload, and their DWs crashed and burned spectacularly. So, even though many companies now have DWs as the stalwart of their IT infrastructures, the concept has a really bad name and is sometimes used as metaphor for failure.

Big data may be on the cusp of going the way of the DW. Today’s pundits are shouting virtually the same accolades at big data analytics as they were about DWs. Harnessing big data can theoretically give you the power to extract better, actionable insights and provide both predictive and prescriptive capabilities. However, big data is also causing headaches at many companies because they simply don’t understand how to effectively exploit its massive potential.

StatsThreeThe numbers don’t lie. Despite the fact that large companies are on an analytics spending binge—IDC predicts that by 2019, companies will spend $187 billion per year on analytics initiatives—these projects are often riddled with complications[1]. Indeed, according to one Gartner analyst, upwards of 85% of big data projects will experience some degree of failure.[2] What’s more, 75% of CEOs feel that they’re doing the best they can with their data, but in reality, only about 43% are actually positioned for success.[3]

How Smart Companies Avoid Failure

There are companies that are doing big data right. Approximately 57% of companies in North America are doing a decent job of wringing valuable insights from their big data analytics projects. What are these companies doing that their competitors aren’t?

They are Data Driven

Companies that are gleaning accurate, actionable insights from the mountains of data they’re wrangling on a daily basis have made the commitment to move away from trusting their gut for decision-making toward relying on the data. It’s hard—especially when those in the C-suite have decades of experience and their gut is telling them one thing and the numbers say another. However, if—and this is a big if—you have analysts who have deep domain experience, you can trust them to interpret the numbers correctly and provide insights that your gut can’t.

They Get the Tools They Need

Analytics RightThere are myriad tools on the market, and every vendor out there will tell you that their tool is the right one for you. It’s not. The tool that’s right for you is the one that will enable you to get the answers you seek. And that’s the catch. To get the right tools, you have to ask the right questions. So, really, it’s not a technical issue, it’s a knowledge—and leadership—issue.

Everyone can contribute to asking the right questions. The C-suite provides strategic direction, and knowledge workers provide deep industry knowledge and business questions. Those questions—and the technology you need to answer them—will determine your infrastructure.

They are on the Same Page—Enterprise Wide

Fully 88% of companies that are doing well with big data say they have a handle on how information flows through the organization. [4] They’re on the same page as to how information affects their job—and how it affects their co-workers’ jobs. That’s important, because an understanding of information flows leads to an understanding of what information you need, and where the gaps in that information lie. When you know the gaps, you can begin to fill them. Those gaps will give you the insight you need to answer the questions you have and glean value from your analytics.

Big Data Doesn’t Have to be a Dirty Word

It’s clear that many companies have a lot of work left to do to realize the potential of big data. Those that are doing it right are reaping the benefits. Those that aren’t are running into some serious headwinds and are feeling the burn of failure. If big data is to live up to its promise, today’s companies will have to tread on the shoulders of those who failed with their data warehouses and learn from their mistakes: become data driven, get the right tools to answer their critical questions, and understand what information they need, and how it flows through the enterprise.

I’d love to hear what you think. Please comment here, connect with me on Linkedin and Twitter, or email me at


nalytics-spending-to-hit-187-billion.html. Retrieved May 16, 2018

[2] Retrieved May 15, 2018

[3] Retrieved May 15, 2018

[4] ibid

Leave a Reply