Pantala

Peak Data

By Leave a Comment

Eventually we will reach a point where there is simply no value to be gained from recording more.

by Dr Thomas Hanna, Founder and CEO of Hypercube Scientific and Pantala’s Business Collaborator

Moore’s law is a famous observation, dating from the 1960s, that available computing power doubles roughly every 18 months. It isn’t true any more. Physics caught up – very small transistors start to be dominated by quantum effects, and energy consumption goes up with chip density. For ten years or so, computers have become wider, not faster, with things such as quad-core chips becoming standard. We’ve also become more concerned about energy consumption – appropriately enough, I swyped the outline of this article on my phone after my laptop gave up during a plane trip.

This graph shows an exponential growth in computer power from 1985 to 2004 at a compound annual rate of over 50%. This growth then slows between 2004 and 2010. Source: “The Future of Computing Performance: Game Over or Next Level?”, National Academies Press 2011

People talk about the explosion in data the same way they used to talk about Moore’s law – with incredulity and cognisance that this is changing the world exponentially (literally, exponentially). I have heard it said many times that 90% of data in the world was created in the last two years. This was also true two years ago, and will also be true in two years. In fact, it turns out the quote is from 2013 or even earlier. It’s another example of exponential growth.

“…(Data growth) can’t continue forever. For one thing, information also has physical limits.”

Data has exploded in recent years thanks to smartphones, the internet of things, and so on. Data capture will continue to increase exponentially while steadily more people adopt these technologies, and while new technologies keep being created to capture new sorts of data. But it can’t continue forever. For one thing, information also has physical limits: it all has to be recorded on something physical, be it ink on paper, magnetic domains on a hard drive, or atomic states. It takes energy to store, read and process data. Also, we’ll eventually reach a point where there’s simply no value to be gained from recording more.

Source: IDC DataAge 2025 Study in Kleiner PErkins Internet Trends 2017, page 132

Some data is worthless. I wouldn’t expect that 90% of the value created from data occurred in the last two years. I have been seeing in new industrial projects a swing from barely thinking about data at all during design, to overdoing it and logging all sorts of things at incredibly high resolution in the hope that it will one day be useful. I have witnessed this swing because part of my work is building simulations (‘digital twins’) of business operations. One value in simulating operations before they even begin is their role in helping you think about how you’ll make decisions day-to-day, and the data you’ll need to do that. I still find myself making recommendations to add a sensor here or there, but also find myself looking at large networks of sensors producing gigabytes per day that add no discernible value. There’s a happy medium somewhere!

“I think my children will grow up and look at how naïve my generation has been about giving out our personal data with the same sort of attitude that my generation takes towards, say, the sexual revolution of the previous generation.”

Dr Thomas Hanna

Personal data is similar. For all the valuable talk about how to regulate internet giants, it is community attitudes that will drive where we end up. I think my children (who are 5, 2, and almost 0) will grow up and look at how naïve my generation has been about giving out our personal data with the same sort of attitude that my generation takes towards, say, the sexual revolution of the previous generation. That was a liberating time of rapid change and advancement. But those embracing the change were at points undeniably naive (‘What? You never even thought of using protection?’). Teenagers are having less sex, and drinking less, than the teens of twenty years ago. They’re more cautious about those things because they grew up in a world that talked about the consequences. But they’re slack and careless about giving away their data because we didn’t start talking about that until very, very recently. They’re happy with the experiences being created in return.

My kids are growing up in a world where iPads, social media, and the sharing economy are as intuitive as computers are to my generation, the first to grow up with a computer in every home. As my kids discover social media, they’ll discover the stories of identity theft. They’ll discover the stories of people who are not really who they claim to be online, and the dangers of that. They’ll discover the stories of people turned down for health insurance or jobs because of personal info that was just ‘out there’. Hopefully they’ll engage with these tools responsibly. And if they happen to look up some of the old articles written by their old man, I hope they don’t groan too much. (Hi kids!).

Eventually, we’ll be capturing enough volume of enough kinds of data, and it will plateau. Then fall, as we cut out some of the trash that deserves to be disposed of and focus on quality and value. We will hit peak data one day, and I’d bet it’s closer than most people think.

Dr Thomas Hanna

Tom is the Founder and CEO of Hypercube Scientific, a boutique consultancy specialized in providing simulation, optimization and advanced mathematical analysis to businesses making substantional investments in their operations, be in in the mining and resources, agricultural or financial sectors.

Hypercube Scientific is one of Pantala’s key business collaborators. To learn more about how we work, please check our About page.

Sources:

Thomas Hanna

Reader Interactions

Leave a Reply

Your email address will not be published. Required fields are marked *