The Death of Big Data

We have become engulfed in phraseology that implies a passive acceptance of forces bigger than ourselves or our ability to manage them.

Big Data. The Cloud. Global this-and-that.

What can mere mortals do in the face of such overwhelming scale - and imprecision.

Which is all very nice for the purveyors of the products and services that purport to deliver these wonders for our edification. But unless companies and the people who attempt to make use of the concepts behind these notions of size and power come to terms with the factors that render them notable, they may find themselves lost or at the mercy of forces whose utility goes largely unrealized.

As the following article suggests, the very term 'big data' may become obsolete. The primary reason is that the power which generated the massive quantities of information its every existence implies may also provide organizations with the ability to modify and manage it. Big Data will eventually become just data. And sources of wisdom will not be denominated by size or scale but by usefulness. JL

Mike Phelan comments in Forbes:

Information technology coins and discards catchphrases with the speed that media dustups-of-the-week come and go in political campaigns. Still, unlike most popular tags, the term “Big Data” has managed to persevere for several years now, and has lately come into its own as the buzzword of the season, if not the year. Those that have Big Data want to celebrate their possession, while those who don’t (as yet) insist they’re getting it or, because of their business focus, don’t need it.

What no one seems to be saying – or recognizing – is that the death of Big Data is imminent. Big Data’s demise is only years, maybe even months, away, and yes, we’ll need a big casket.

Just what is Big Data?

If you’re unfamiliar with the term, Big Data is used in the singular (Latin scholars, sorry!) and refers to a collection of data sets so large and complex, it’s impossible to process them with the usual databases and tools. Because of its size and associated numbers, Big Data is hard to capture, store, search, share, analyze and visualize. The phenomenon came about in recent years due to the sheer amount of machine data being generated today – thanks to mobile devices, tracking systems, RFID, sensor networks, social networks, Internet searches, automated record keeping, video archives, e-commerce, etc. – coupled with the additional information derived by analyzing all this information, which on its own creates another enormous data set. Companies pursue Big Data because it can be revelatory in spotting business trends, improving research quality, and gaining insights in a variety of fields, from IT to medicine to law enforcement and everything in between and beyond.

So why is such a useful development in information science likely to die soon?

Well, for starters, even the very definition has a short lifespan. The sizes associated with Big Data keep changing – usually getting larger every year – with 2012 data sets ranging from a few dozen terabytes to many petabytes of data. Why does it keep growing? A simple example tells the tale. Reportedly, Facebook already has 40 billion photos in its user base. You can bet that Apple-philes will be snapping new images with their iPhone 5 like crazy, and uploading them directly to Facebook with a simple voice request to Siri. Despite this unprecedented data growth, it’s time to buy your Big Data sympathy cards now.

Experts Predict an Early Demise

According to a July 2012 report titled “Hype Cycle for Big Data, 2012,” Gartner does not believe that Big Data as a “hyped term” will be in vogue for very long: “Unlike other Hype Cycles, which are published year after year, we believe it is possible that within two to three years, the ability to address new sources and types, and increasing volumes of data will be ‘table stakes’ – part of the cost of entry of playing in the global economy. When the hype goes, so will the Hype Cycle.”

In September 2012, commenting on the volume, velocity and variety of data in both commercial and consumer networks, Morgan Stanley insisted that as opposed to Big Data “the paradigm shift we are seeing in data management is more about giving customers the technologies they need to store and analyze ‘any’ data set – any type of data, any size of data, for any type of user, and in any timeframe.”

IDC estimates that the amount of data managed by enterprises will grow by a factor of 50% over the next decade, and that 95% of this data is multi-structured in nature and is increasing at an exponential rate that far outpaces the growth of structured data. Morgan Stanley’s report insists that “Big Data” is soon to become “Any Data,” noting, “Based on IDC IT market sizing, our data points to an incremental $115 billion in data-related spending through CY15.”

Translation for all that report-speak is this: Big Data is fast becoming the norm. The trend is so indisputable that in less than three years large data sets will be so common, only those way behind the curve will even think of using the term “Big Data” regarding their database resources. Let the epitaph read: R.I.P. B.D.

Who Gets Big Data’s Legacy?

The biggest problem with Big Data is, well, its “bigness.” Companies are pouring tons of money into efforts to cope with the massive influx of available data right now, and some specialized sectors of IT are designing solutions to handle the mountains of information in ways that make sense of it all. Morgan Stanley reports that, not surprisingly, enterprises manage on average over 20 times more data than small and medium businesses, and 50% of enterprises have a Big Data initiative versus 20% for SMB. In other words, companies are still in the very early stages of coming to grips with today’s huge amounts of data.

Once Big Data becomes Any Data, and every company has figured out how to handle and make use of the vast reserves of information available, the winners will be easy to spot. They will be those companies that early in the game focused on ways of handling data acquisition, analysis, storage, sharing, and visualization. They may not be the companies that are re-inventing themselves to deal with the hype of the moment, Big Data, but rather those companies that have been handling and analyzing data in great volumes and at great velocities all along.

Beware the Slag Heap of IT Terms

Here’s our advice: accept the fact that “Big Data” will soon become the norm, and remember that no one tags a name on what is the norm. Bandy the term around the lunch room if you must, but when you move into the corporate planning room, turn your attention to those firms who can help you cope with data in any amount now.

If you can handle Any Data, you can handle Big Data. And if you get in the habit of not referring to “Big Data” sooner rather than later, you won’t be one of those people who still uses slang from years long gone and think they’re “hip” in the twenty-first century.