In 2008 the World Economic Forum reported the beginning of a new age, that of big data. Enter the zettabyte, a 9.57 trillion gigabytes unit of information that we now had available for processing.
“Out with every theory of human behaviour, from linguistics to sociology. Forget taxonomy, ontology, and psychology. Who knows why people do what they do? The point is they do it, and we can track and measure it with unprecedented fidelity. With enough data, the numbers speak for themselves.”
The article is over ten years old now; a piece of writing from the era before Google dropped it’s ‘Do No Evil’ police from its code of conduct.
It was written before politicians here in the UK and elsewhere started the desecration of fields of expertise across health, finance, humanities and anyone who disagreed with their policites.
Perhaps the article came from a place of hope but really, what has big data done for most of us?
The original Wired article mentions science as a potential benefactor of the ‘correlation is enough’ philosophy that sees the search for causation as unnecessary waste.
Average temperatures continue to increase, and if you’re not feeling the heat just by looking at the pictures of the wildfires in Australia and the USA, then you’ve got an icier constitution than the melting Antarctic.
On top of that inequality continues to grow and because we’re only mid-cycle through the COVID-19 pandemic, we’re yet to see the outcome for lower economic groups.
Whatever the promise of big data actually was, it hasn’t landed with most of us.
So why do we look to big data?
The idea that we can buy a set of correlations and turn them into a theory or a product appeals.
We don’t lose time or money investigating things that don’t work.
We’re buying a piece of confidence that ordinarily is gained through trial and a whole load of error.
And of course when the case studies work for big data they are huge. Think of the astronomical profits of Amazon and the celebration of Jeff Bezos crossing the 200 billion dollar net worth recently.
But the snake oil here is that data only works well when you have a problem you set out to solve. You need to be in search of a hypothesis before making use of data; doing it the other way round is not efficient.
The numbers can’t speak for themselves. They’re just numbers.
In our current data quagmire real damage is being done to the fabric of our society by our inability to transform information into knowledge before it can be disseminated as misinformation and propaganda.
He reports a 60–85% failure rate of data set implementation for businesses. That’s not the kind of narrative that we’re used to hearing in relation to data.
The real gold at the heart of any successful project is going in with a cause, or a problem to solve.
Looking around at some of the current, or new success stories, we find this mindset is driving growth and results where merely analyzing data cannot.
The authors of these other stories are not going in via a direct-to-data route. They a problem-solving mindset that starts with a quest and uses data to back up their theory. Rather than letting the numbers speak for themselves.
Of the public success stories, Netflix is one of the biggies on circulation at the moment, and you can’t argue with the numbers or the relevance in COVID-19 environment.
Since its inception in 1997 as a direct mail DVD service, at a time when few households had evolved beyond VHS rental, it has grown into a giant of data-driven customization.
Bringing personalized recommendations based on your likes and watching patterns and introducing entirely new genres and markets to viewers.
Following on from the release of his book, That Will Never Work Marc Randolph discusses the intersection of ideas and data on developing a new product in an interview with Storius Magazine.
He believes that data needs to follow on swiftly from a new idea but that the initial point of inspiration comes from following your intuition and ultimately trying to solve a problem.
There’s a legendary myth about Randolph’s Netflix co-founder dreaming up the idea after receiving a $40 late fee from Blockbuster and musing over a world of rental entertainment without late fees. The story has not been claimed as true, but it does fit the bill with the evolution of an idea.
Randolph goes on to say that problem solving is the true genesis of a fruitful idea though an element of luck will propel you forward.
“Ideas come and go — but problems are there forever.”
This human-centered focus strategy has remained in the work of Netflix CEO Reed Hastings and his talent acquisition strategy. He invests in the superstars of the thinking world; rather than relying on technology or data alone. He recruits the people who can take the creative and risk-taking decisions because the numbers alone cannot tell this story.
Pierre Omidyar, mega philanthropist but more commonly known as the founder of eBay was on a mission to empower people to define their own value when he created his iconic on-line marketplace.
He tells the story of trying to invest in IPO for a gaming company and being refused shares at a lower price because he was a regular person.
The problem he set out to solve was empowering people to benefit from the theory of efficient markets, instead of being locked out by bigger players.
eBay, a market where people could set their own value was born.
Sasha Haco, young entrepreneur with a degree in natural sciences and a PhD in theoretical physics created a seed-funded AI moderation app to trawl the internet and complete necessary task that takes its toll emotionally on the humans that currently do the job.
She was a colleague of the late and much-missed Stephen Hawkings and was listed in a Beauhurst report of the UK’s Top 10 Female Entrepreneurs Under 30.
She talks to Tech Crunch about her mission to make the internet a safer place.
“Regulations are responding to this crisis and putting increasing pressure on platforms to deal with harmful content and protect our children from the worst of the internet. But currently, there is no adequate solution”.
Harvesting Imaginative Solutions
We’re always told to chase the data as entrepreneurs or business owners. Do your research. Get the numbers. But what’s the question? How can you drive change with numbers alone?
And how do we find our power? It’s not in a series of numbers or correlation data. We find our solutions on the road; in the quest at the end of a road of discovery.
Creative solutions can’t be packaged or marketed in the same way that access to data can.
From imagination comes intuition, hope, problem-solving, inspiration and sometimes for those who persevere this can translate into a profitable business.
Whether the ‘data will fix it’ mentality comes from a place where imagination has been forgotten, or it’s of a more sinister marketing tactic to try and sell you something you don’t need — , there is another, better road.
Inequality grows, while negative climate change grows, and data grows too.
There’s a correlation you don’t need a computer for but without a mission, how are we going to change this? What’s the question we’re trying to solve as consumers or entrepreneurs? What’s the big mission?
While the trends are worrying, we can look around to another artform to provide ideas and guidance. The art of storytelling. Why storytelling? Because all good stories start with a quest and good quests, make for good innovation and promising innovations make for successful businesses.
So if a data package or database doesn’t work what will? If you’re a writer or entrepreneur, you should start with a question.
Like Pierre, like Reed, like Sasha, — find your mission and push it forward. Don’t have a mission in life? That’s okay but don’t believe everything about data.
If you’re an entrepreneur or trying to do something different with your life then where do you find your mission?
It starts with a hunch, conversation or a smile you receive from a stranger when you’re walking your dog in the park.
It comes from life and stories and hope.
I’m a writer and storyteller, investigating the noise and trying to create some of my own too. If you’re interested in receiving a weekly roundup of similar articles, you can join my newsletter here.
Thanks for reading.