The Importance of Context in a Data Rich World

Could it be that the age of big data is already over?

Not that data has become less important or a less significant driver of decisions, choices and overall behavior. But that data has become like electricity and automobiles and televisions and computers.  As in, 'ok, it's an important part of our lives. It's everywhere. Now what do we do with it?'

So we might call this the second phase of the Big Data Age. The Age of Context. Because if we take data as given, it's about as useful as electricity without a device to plug into it or about as useless as a car without a road.

Lots of enterprises are producing great reams of interesting data and even some rudimentary interpretation: Google, Amazon, Facebook (yeah, we know: debatable but they're trying, even if their goals are not always consistent with those of their customers). But as the following article explains, without further, intensive and imaginative examination of sources, uses, goals and outcomes we will continue to take it for granted, which means those who produce it run the risk of taking us, the users, for granted. And that would be a missed opportunity for all involved.  JL

Om Malik comments in GigaOm:

“Data, when its put up in front of you as a number, it gets stripped of the context of where the data came from, the biases inherent in it, and the assumptions of the models that created it,”

I am known to go to great lengths and even greater distances to meet great bootmakers and shoe stores of some repute. Most people make vacation itineraries around the idea of seeing ruins or really old buildings. I prefer shoes and fantastic food (in that order) to be my guiding light.

It must be because as a child, I grew up with only two pair of shoes — one for school and another for play. Eating out became an option, only later in my teen years. Like, I often say, what you don’t have as a child defines you more than what you actually do have. So perhaps that is why when I go to a foreign city, I finish my work and find a few great restaurants. And then walk off the good food by walking to some little known shoe store (for rest of the world) or a cobbler of some repute.
A friend, once finding out that I was going to be visiting Tuscany recommended a shoemaker, for he lived a few miles from where I was going to be making a temporary residence. He said it was perhaps about 25 miles to the ancient city of Siena, which is well known for its annual festival, Palio di Siena.
Having gotten used to the American roads, I roughly equate 25 miles to about 30 minutes — thanks to fairly straight and wide freeways, especially in the midwest and western parts of the country. The speed limit too isn’t too bad and folks traverse great distances at a pretty decent clip. So when I think of 25 miles, I often think about the distance from San Francisco to Redwood City, which on a decent traffic day is about a 30 minute drive down US Highway 101.
With that in mind, I was looking forward to this quick drive to Siena. When, I looked at the Google maps at the start if the journey, Google Maps said that it was going to take about 57 minutes to travel 40 kilometers. In reality, the trip ended up taking about 80 minutes. The giant Google machine hadn’t really accounted for quite a few things machines aren’t supposed to understand — because they aren’t quantifiable.
For instance, as a US driver, you tend to get a lot more cautious on curvaceous and hilly roads, compared to Italians who throw around their little Fiats like go-karts on a racing track. Google didn’t account for one getting stuck behind a neon-colored lycra festooned bicycle group for ten minutes, before they could maneuver and give you room to pass. The machine also didn’t know that overnight some parts of the road has just crumbled away and well, driving became a lot more perilous.
****

1. In this data driven world, how do we (and software and devices) deal with unexpected divergences between “the data” and the real world? Do we become so dependent on data-driven algorithms that some stale data can cause major havoc? How do we gracefully recover from such situations?
2. How does the new world of data-driven intelligence not just provide quotidian services that make our lives more efficient, but create moments of serendipity that make our lives more joyful?

***
That picturesque drive (via SP 222, one of the most beautiful roads in Italy) through the Italian wine country was a great reminder of how much, rather, how little computers and data have context, both temporal and spatial. And it brought up two quintessential questions about the future of our data-shaped society.
The same distance means something else in a bus, in a straight line, or in my case, on SP 222, which like a silver snake slithers through the rising (and falling) contours of Tuscan hills. 40 kilometers in the US is very different from 40 kilometers in Tuscany or in Germany. Same 40 kilometers means differently to an American in Germany or an Italian in America. Similarly, 15 degrees centigrade is positively winter in New Delhi, but back in San Francisco that passes for a lovely day.
Both are a good reminder that context is everything and that context changes with every bit of additional data points. As data starts to shape our expectations and perhaps our experiences, one has to think about the role of randomness in providing some context to that data. How will data driven experiences — autonomous cars for example — account for the unknown, like the crumbled side of the road.
One of the reason I am fascinated by Google Maps and apps like Foursquare is not because they solve a very real problem, but instead, for me they are a living test bed of an Internet that is shape-shifting in real-time, is data rich and hyper-personalized to such an extreme that it can predict what comes next almost automagically. It is a network that uses connectivity to its extreme and offers the impossible.
Traffic and calculating its various permutations is actually a unique problem, though I suspect Google is most likely to be upto the challenge, thanks to its wide array of technology assets and its desire to play a role in the future of transportation, whether through autonomous cars, ride shares or some yet unknown experiment. A new class of network is emerging that invisibly strings together moving objects (cars, trucks, buses and other vehicles) giving them a relative context to each other.
Google owns Android and as such it has the ability to get a ping every so often from a moving Android device and can easily deduce speed. You can use this data to conclude the relative speeds and distances between various vehicles as well. Google, can also deduce if a US phone-number based device is being used in say Italy, and thus use that as a variable when calculating the speed and time to travel for an American driver. Data from Google-owned Waze can add even more nuance to driving conditions. Google recently bought a cube-sat company that will give its satellite eye view of the world on a near real-time basis. It is easy to foretell Google and its impact on the transportation business. Maps + Waze + Satellites + Cars = data that powers CarOS. In the future, carmakers pay a license fee for that CarOS.
I often think about the data rich world and wonder if we have to think less like scientists and more like data artists, who take every little detail into account. An extra stroke of green or a different shade of green can turn a grassy knoll into a lush meadow, or a woodland, a summer morning into rainy afternoon. In March 2013, I gave Uber a lot of stick for being soulless with their data. Earlier this summer, Uber’s Ren Lu in a blog post shared details of how Uber was trying to make predictions about their customers’ destinations. “Extensions of this project involve building more complex priors and likelihoods,” Lu writes. Those are the brushstrokes that change the perspective.
Where you get dropped off, where you subsequently order an Uber again perhaps in two hours and correlated with a local database, and the frequency of visits can give Uber a much better idea on predicting your next ride. Every additional piece of data for me is like that extra stroke of paint. The thinking here can’t just be more data — art is not about creating perfection, but instead it is about imperfect perfectness. Art is all about perception, reminding me of what Jonah Jones (who has spent past few years on Google Maps) once said: “There are no absolutes, its all perception.” Today’s world of data is about absolutes, often faulty and presumptuous.
Conversations today center about data analytics — where all people are looking to do is sort through the data they have, look for hidden trends, try and predict behaviors and then optimize. It is very cut-and-dry, antiseptic and without soul. I guess it is a pet peeve of mine — I keep bringing this again and again.
“Data, when its put up in front of you as a number, it gets stripped of the context of where the data came from, the biases inherent in it, and the assumptions of the models that created it,” said Sean Gourley who started Quid, a San Francisco-based data analytics company. “Data scientists are presented with a set of parameters to optimize over, yet they don’t take the time to step back and say, should I even be optimizing this at all. We need to step back from the scientific notations and start thinking of it as data intelligence.”
In the data rich world of tomorrow, all incoming data should come with a question it answers, adding more context and thus constantly enhancing the experience. The mapping apps are going to be at the forefront of this data experience movement. Sure in order to create these constantly updated, constantly refreshed and reshaped experiences, you need a lot of oomph. Widespread server farms, data processing engines and ability to get many different data feeds will be key to that future.
But this is not just about technical prowess, but is also about ability to weave together many different strings. In Florence, the city where SP 222 starts, I relearned the importance of polymaths. The marriage of art, science, divine and beauty were the gift of Florence, the cradle of renaissance. From that perspective, the data factories of today — Google, Amazon, and Facebook — have an opportunity to reshape our expectations and experience.
The number rule in my book is — Internet taketh & Internet giveth aka whatever is lost because of Internet is also replaced by a wonderful new gift from the Internet. If technology today has reduced actual humans to antiseptic clicks, users, engagement minutes and made everything “data” then the same technology has the ability to rekindle what is being lost in this data-infused world: subtle magic of discovery, so subtle that it is near invisible and yet brings a hint of a smile, unintentionally.
I am a sucker for maps — in fact I am a sucker for maps that tell me about the past. They are works of art — the colors and the shades, the thickness of the pen, the typography, make them much more interesting that essentially tools of navigation. They bring a sense of place and add the joy of journey. David Churbuck calls it “the metaphysics of mapping, of cartography, and of providing the stranger with a guide to the familiar.”
Google’s vision of the future should be progressing towards that metaphysical. Armed with sensors, data and live connection to the Internet, a map shouldn’t just be a map — it should a storyboard for journey yet not undertaken. It has to be essential for Google’s vision of the future — where they marry the social data of Waze and traffic reports, a future where cars will lack the personality of their predecessors and the journey will unfortunately be more (for the lack of a better word) plastic.
The sheer joy of the Internet is the speed with which choices multiply. The sheer agony of the Internet is the speed with which choices multiply. That is why we need this new kind of creativity. It has to figure out a way to nudge us to new things, and enable wonder and discovery. What would be cool — is not just the efficient data, but also small details that tell us about things we might find if we went off the beaten path, took a detour or two and let joy of discovery be the overriding factor. If this was a feature in Google’s CarOS, it would bring back some of the joy we associate with the experience of driving, and the serendipity. Of course, everything would be filtered through Google’s algorithms, but as long as Google doesn’t completely lose its humanity, we might just have to suck it up and deal with the world which is much bigger, larger and complex.
There is such a huge opportunity and also a massive risk of living in a data enriched world. Risk is because going to be left to the devices of those who have a singular point of view. I am often perturbed by the idea of someone using data to define our destinies. It is an opportunity, for it allows science and art to embrace curiosity and start to define a new kind of creativity.
What is that creativity, you might ask? To be candid, I have been thinking about it. I am first to admit, I don’t have any answers. I do know that it is not a filter bubble. I also am confident that it is not personalization through marriage of handful of numbers. I also know that it is subtle. It is also not more, but instead it is using more of everything and yet showing us less, showing when needed, without even knowing we need to see it. It will come not from science, but from imagination.
***
It is the kind of creativity you see in a seamless whole cut oxford shoe with no seams. The perfect leather, the clean simple look, the immaculate shape and quality. It is incredibly hard and the shoe itself is the detail. It tells the story, all the complexity simply hidden from the naked eye.
The quest for creativity is what got me to take a break from my vacation, and visit the wonderful shoemaker. We discussed philosophy, we talked about greed and ambition and more importantly we talked about shoes. Do shoes have a soul? Later, I fussed about the leather choices, the colors. In the end I ordered a pair. It is a shoe, like any other. But it will never be like any other. It is mine — it is unique, but more importantly, it has a story.
When I will look at those shoes, it will remind me of the drive, the quaint little shop, smells wafting out of trattoria next door and it will take me to a place that will make me smile. I would have reached my destination.That’s my kind of personalization.