Why does data need to have sex?
Wednesday, July 2, 2014 at 9:00AM 
Data needs the ability to combine with other data in new ways to reach maximum value. So data needs to have the equivalent of sex.
That's why I used sex in the title of my previous article, Data Doesn't Need To Be Free, But It Does Need To Have Sex. So it wasn't some sort of click-bait title as some have suggested.
Sex is nature's way of bringing different data sets together, that is our genome, and creating something new that has a chance to survive and thrive in changing environments.
Currently data is cloistered behind Walled Gardens and thus has far less value than it could have. How do we coax data from behind these walls? With money. So that's where the bit about "data doesn't need to be free" comes from. How do we make money? Through markets. What do we have as a product to bring to market? Data. What do services need to keep producing data as a product? Money.
So it's a virtuous circle. Services generate data from their relationship with users. That data can be sold for the money services need to make a profit. Profit keeps the service that users like running. A running service produces even more data to continue the cycle.
Why do we even care about data having a sex?
Historically one lens we can use to look at the world is to see everything in terms of how resources have been exploited over the ages. We can see the entire human diaspora as largely being determined by the search for and exploitation of different resource reservoirs.
We live near the sea for trade and access to fisheries. Early on we lived next to rivers for water, for food, for transportation, and later for power. People move to where there is lumber to harvest, gold to mine, coal to mine, iron to mine, land to grow food, steel to process, and so on. Then we build roads, rail roads, canals and ports to connect resource reservoirs to consumers.
In Nova Scotia, where I've been on vacation, a common pattern was for England and France to fight each other over land and resources. In the process they would build forts, import soldiers, build infrastructure, and make it relatively safe to trade. These forts became towns which then became economic hubs. We see these places as large cities now, like Halifax Nova Scotia, but it's the resources that came first.
When you visit coves along the coast of Nova Scotia they may tell you with interpretive signage, spaced out along a boardwalk, about the boom and bust cycles of different fish stocks as they were discovered, exploited, and eventually fished out.
In the early days in Nova Scotia great fortunes were made on cod. Then when cod was all fished out other resource reservoirs like sardines, halibut, and lobster were exploited. Atlantic salmon was over fished. Production moved to the Pacific where salmon was once again over fished. Now a big product is scallops and what were once trash fish, like redfish, is now the next big thing because that's what's left.
During these cycles great fortunes were made. But when a resource runs out people move on and find another. And when that runs out people move on and keep moving on until they find a place to make make living.
Places associated with old used up resources often just fade away. Ghosts of the original economic energy that created them. As a tourist I've noticed what is mined now as a resource is the history of the people and places that were created in the process of exploiting previous resources. We call it tourism.
Data is a resource reservoir like all the other resource reservoirs we've talked about, but data is not being treated like a resource. It's as if forts and boats and fishermen all congregated to catch cod, but then didn't sell the cod on an open market. If that were the case limited wealth would have been generated, but because all these goods went to market as part of a vast value chain, a decent living was made by a great many people.
If we can see data as a resource reservoir, as natural resources run out, we'll be able to switch to unnatural resources to continue the great cycle of resource exploitation.
Will this work? I don't know. It's just a thought that seems worth exploring.
Reader Comments (9)
Very interesting thoughts. Some thoughts:
* I would extend "data" to "information".
* information can also represent knowledge, pure factual data or refined data mined by processing existing data (e.g. stock indicators, ..). Processing data requires knowledge and a processing system => money
* Service users are not the only source of information
* Data value frequently decreases with age (e.g. market prices, weather information, traffic data).
The ideal would be a large exchange/huge system providing access to streams of data fed in by data vendors/data refiners.
on a second thought some random remarks:
* replication of information (data) is nearby free, so there is no classical demand/supply market. As soon a required piece of information is available, all demand is satisfied (except for realtime data streams)
* replication of data induces latency. In some market areas this creates a market as information is priced along its age
* so a data-trading platform would find a (formal) way to quantify demand for a specific information. Once demand is high enough a data producer will satisfy demand reactively.
Information is very much NOT a conventional resource. A resource's value is determined by supply and demand. Information can be spread easily at no cost so demand drops towards 0 as soon as it is released. The usefulness of information (achieved by spreading) is completely at odds with the profitability.
Encumbant industries are trying to enforce artificial constraints on the sharing of data through copyright etc, but this is a very poor fit. By trying to treat it like a physical item we are missing out on the vast benefits and this needs to change. We have to find a model where collecting or processing data is rewarded enough to make it worthwhile but without blocking others from taking the data and adding their own value to it (which may well be at odds with the creator's wishes but in the interest of society).
How can we determine which value is important, how money can trickle back, etc.? That is a hard problem. But until we solve it the status quo will continue to block adding value and the true worth of mixing data that your article talks about.
I still think there is a difficulty being skirted around here and that is lack of property rights suitable to make this data free trade/exchange viable. As I commented on the previous blog post, it seems to me that data privacy is the data analog of property rights. If we don't have data privacy (i.e. property rights) at an individual level, what is being proposed here seems to be out of reach.
Ok, now I get the metaphor of data mating with other data, the baby would be an aggregation. The more data you can get from the more sources and the quicker you can aggregate it 1000 different ways is the path to maximize the ecosystem's value. So sex was a conservative way to phrase the metaphor ;)
The basic premise of this system is you try to get a lot of applications to share their data with the system, and put a sell-price on their data. If their data has some value, other apps will need it enough to pay for it. A mature market-place would allow middle men to aggregate various streams of data and resell the results. End customers buy the raw data or the processed data aggregation, the money trickles backwards to the producers and middle men and the system takes its cut.
This concept is so simple and straightforward, it is hard to punch holes in it.
So why doesn't it exist? Are the big companies too smart to give up their share of the data market, are they the equivalents of the Robber Barons? Does this concept need to find its first end to end producer to consumer chain that pays enough to fund the operational costs and R&D such a system would require? Are the legal/copyright/privacy hurdles too high?
I feel like I must be missing something, because this is a very powerful yet simple concept and the first I hear about it is via Todd daydreaming in foreign fishing villages ;)
Tristan, digital property rights will always be problematic, no getting around that. But is that a reason not create a system around data?
Your property is never truly safe as it can always be stolen, but we think a police function along with vigilance is sufficient.
And in another domain, if you are a farmer, for example, there's always an expected loss due to weather, predators, etc. So the rule of thumb is always to plant 20% more than you need.
YouTube seems to do a good job handling digital rights and splitting revenues amongst right holders.
Perhaps we create a ditcoin that uses a block chain for accessing data?
I think we can create system that while not perfect does allow for an exponentially growing market. What would it look like? There are a lot of smart people out there who no doubt have good ideas.
"dreaming in foreign fishing villages" I like that :-)
Its all well and good until your data comes home pregnant.
Good thing I store everything in a data whorehouse.