CRF Blog

Fix This: Big Data

by Bill Hayes

In Fix This: Big Data, a feature story, Bloomberg Businessweek interviews four big-data experts on how they would fix medicine, education, and other areas.

Consultant Andreas Weigend on Big Data Refineries

You’ve compared big data to oil.

If you find raw oil in your backyard, it’s probably not that useful. You need to refine the raw materials in order to get something that people need. Raw data is not that useful, either. Examples of data refineries would be Amazon (AMZN) or Google (GOOG). A big difference, of course, is that we’re not going to run out of data anytime soon, as far as I know. So in terms of pricing, there are interesting implications of information products vs. oil products.

The process of drilling for and refining oil has led to all sorts of environmental and political problems. What’s the parallel for data?

If you want to push the energy metaphor, it is more like a nuclear power plant, where the question is are we willing to deal with the risk of nuclear accident, which is very unlikely but when it happens: big problem. A nuclear accident would be like late last year when data on a third of all credit-card users in the U.S. leaked.

Seems like many of the benefits are enjoyed by the companies you describe as refineries, as opposed to their users. How do we make sure everyone benefits from big data?

I think that what has happened with Apple’s (AAPL) App Store will happen in the data economy, where companies will build services on top of the raw material. There is value in having an app store where another party uses data to provide apps for the consumer and shares revenue with the data company.

Doesn’t this create very powerful gatekeepers?

Yes. At the same time, a lot of the valuable research has moved to places like Amazon or Facebook (FB), and if we try to take antitrust action against them in the best interest of consumers, we should look at the second-order effects. [more]