Big Data: Friend or Foe to Creativity and Innovation?
Mark B. Mondry, Phase M, LLP
Originally published: 2013 (PDMA Visions Magazine • Issue 4, 2013 • Vol. 37 • No. 4)
Read time: 5 minutes
Who’s afraid of big data? Well, I must admit I was—at least, as big data relates to creativity and innovation. Like many others who reside in, or at least brush up against, the creative community, I considered the rapid emergence of big data as a threat to creativity in the innovation and new product development space. My fear was that big data would replace human decision making with data-driven algorithms, displacing human centered imagination with micro-measured digitalization of the consumer experience. Would we make all of our business decisions by looking into the rear-view data mirror? Would we lose the freedom of inspirationally gazing forward and envisaging the future? Apparitions of creative stagnation danced in my head.
I now realize that my fears were embedded in an incomplete notion: that big data is primarily about capturing what has happened in the past. Data must be generated before it can be captured, aggregated and analyzed. Thus, data represents the past tense. How can this empower creativity? Experience has taught me that predicting the future based only on what has happened in the past is a recipe for disaster. I was, therefore, apprehensive about the notion that big data could fuel innovation. Fortunately, I have since evolved.
"Big Data vs. Little Data According to Jerry W. Thomas, the truth is that the solution to marketing and business problems—and the identification of strategic opportunities—often lies in the realm of little data, not big data. You don’t have to boil the ocean to determine its salt content. You don’t have to eat the whole steer to know it’s tough."
Learning: The Entrepreneurial Method
Big data is not just about capturing the past. The true power behind big data is real time experimentation and analysis, and the learning that can be gained from it. The ability to capture, aggregate, store, communicate and analyze colossal quantities of data, and more importantly filter out the noise from these vast collections of data, allow companies to test hypotheses, challenge assumptions and change direction more quickly than ever before. This concept, albeit at a much smaller scale, might sound familiar to you, and it should. It comes from the contemporary startup world.
Entrepreneurial startups have demoted the traditional static business plan in favor of active learning and fast iteration. The process involves framing a series of hypotheses around a proposed business model,1 getting “out of the building” to gather real-time customer feedback on the hypothesis by testing simple prototypes, and quickly “pivoting” to alternative opportunities when things don’t seem to be working out. In startups, this cycle is continued until a successful business is identified that can be scaled. This deploy-test-adjust process of rapid iteration forms the foundation of the lean startup2 model. Big data provides the opportunity to scale this basic concept into large, established enterprises.
I now think of it this way: Big data allows the aggregation and real-time analysis of the vast amount of information that is generated as mature businesses operate, try new things and experiment. Big data can propel rapid iteration in large companies, allowing them to become more innovative, and to behave more like nimble startups. So, I have evolved to appreciate that big data and creativity can co-exist.
Big Companies Want to Learn, Too
The amount of information being generated daily as companies continuously interact with their community of upstream and downstream enterprises, and their potential and actual customers, is astounding. Internal and external operational and marketing data combine with the flow of social media interactions, call center voice data, radio-frequency identification (RFID) location tags, financial and economic data—as well as the information gathered from an exponentially growing network of sensors embedded in industrial machines, transportation vehicles, appliances and communications devices—to form a tsunami of data. This, of course, is the challenge. There is just so much information being generated, it is very easy to drown in the noise.
According to recent research, more than 90 percent of all the data in the world has been generated in just the last two years.3 Every month, approximately 30 billion pieces of content is shared on Facebook, while Visa processes approximately 4.6 billion credit card transactions. Google alone processes about 24 petabytes (24,000 terabytes) of data every day. More than 48 hours of video content is uploaded on YouTube every minute. These activities and the content inside of them generate data, and the volume and velocity of the resulting data flow is increasing. The opportunities to acquire and leverage this data are expanding even faster than we can react to them.
Big data early adopters have largely been focused on consumer advertising. In fact, the revenue models of companies like Facebook and Twitter are dependent upon the big data algorithms and methodologies that will help them provide micro-segmented audience targeting for advertising and personalized consumer offers. If your industry is consumer goods or services, this is fantastic. But what if your business isn’t focused on selling consumer products?
For an increasing number of companies, the key value proposition in the future of big data is that learning can be generated quickly and at low cost. The most valuable learning does not come from analyzing static collections of data, but in accessing and analyzing streams of information in real time. It is this information flow that generates true insight, like amplifying the concepts of design research to a previously unimaginable scale.
Thus, big data can increase the opportuni-ties for creativity. The ability to process and analyze data during an actual event (healthcare utilization in a hospital or a national product rollout, for example) allows organizations to actually improve the outcome. The dynamics truly change, from discrete phases of “act, measure and respond” learning, to realtime testing of hypothesis and continuous learning. This process lowers the costs of hypothesis testing, allowing companies to be more creative and innovative. Organizational learning happens at a higher frequency and faster clock speed.
We All Have More to Learn
The possibilities of big data have a scary side, too. Will the capture and analysis of all this data yield benefits for all, or will it merely amplify the fears of an increasingly intrusive world? The privacy, security and intellectual property issues surrounding big data are real and must be addressed. We need to remind ourselves, however, that the big data industry is still relatively embryonic. The ability to cheaply capture massive amounts of information and rapidly analyze it in a productive way is a recent product of cheap and scalable digital storage combined with recent software tools like Hadoop, MongoDB, and Apache Hive and Cassandra, among others.
I believe companies should resist the urge to race into big data just because it is new and exciting. There is significant hype surrounding the big data industry, and the hype fuels the action-bias commonly found in most companies to quickly embrace the next big thing. Before launching any big data project, ask the single most important question: why? How will big data, properly deployed, help your business? How will you address the numerous privacy and security issues that coexist with the acquiring such information flows? It is easy to spend vast sums of money and resources on poorly defined big data opportunities that flounder and fail to achieve expected results. The software tools continue to evolve by necessity, and there is a real and growing talent shortage of knowledgeable people who can make sense of the flood of new data being collected and isolate insight from noise.
But Don't Let Fear Get In Your Way
Yes, there is much we still need to learn about big data, but there is even more that we can learn from it. Big data is merely a byproduct of the information generated from business operations and the interactions between businesses and people. This information has always been generated, but it is only recently that we have developed the tools and methodologies to capture it and analyze it in an economical way.
My hope is that big data can be deployed to substantially enhance creativity in new product development and innovation. It will take creative minds to figure out how, and creative minds to make it all work. Therefore, I believe big data and creativity are intrinsically codependent.
About the Author
Mark B. Mondry, NPDP, Managing Partner, Phase M, LLP, works in a variety of industries to identify and establish international collaboration opportunities around the creation and commercialization of innovation. He is an engineer, patent attorney and certified licensing professional (CLP) focusing on Asia-United States business transactions.
References
1Strategic tools such as the “Business Model Canvas” are being widely embraced by startups to visual-ize business model hypothesis. A. Osterwalder, Y. Pigneur, A. Smith, and 470 practitioners from 45 countries, Business Model Generation, Wiley (2010).
2The Lean Startup model was developed from the foundations of lean manufacturing. Ries, Eric, The Lean Startup: How Today’s Entrepreneurs Use Con-tinuous Innovation to Create Radically Successful Businesses Crown Publishing (2011). The concept has since been embraced by the startup community and further refined. See, for example, The Lean Entrepreneur, B. Cooper, P. Vlaskovits, Wiley (2013).
3SINTEF (2013, May 22). Big data, for better or worse: 90 percent of world’s data generated over last two years. ScienceDaily. Retrieved October 20, 2013, from www.sciencedaily.com/releas-es/2013/05/130522085217.htm