Big Data 101: The Gold Mine of Internet

Big Data: the concept is expeditiously becoming the buzz-word around the corporate houses and governments. So, what exactly is Big Data? Let’s say after watching a movie, you commented on Facebook or tweeted on Twitter. Your friend does the same, and so does your neighbor. Now imagine millions of people around the globe reviewing, commenting, tweeting, blogging, clicking pictures, all about the same movie over the internet, makes it a data worth billions of bytes. This data spread across the internet, unstructured, is called Big Data.

In recent years, the internet boom has given birth to new terms and abbreviations in data management, like never before. Technology Magazine Baseline states that “90% of world’s data has been created in the last 2 years.” The internet is at the dawn of approaching data equivalence of Yottabytes (1,000,000,000,000,000,000,000,000 bytes!), and it would be no surprise when the internet dictionary gets yet another word after that.

For small businesses it might be a steep ladder to analyze such humongous data for their benefits, yet according to McKinsey – a business using big data to the full could increase its operating margin by more than 60%.

What’s so Big about Big Data?

The name itself is axiomatic. Big Data is the Burj Khalifa of internet. Let’s see some charming analogies to understand the enormity of data in this technologically shrewd epoch.

  1. If you stacked a pile of DVDs on top of one another until you’d reached the current global storage capacity for digital information – about 3500* Exabytes – it would stretch more than 1 million km beyond the Moon.
  2. Every hour, enough information is consumed by internet traffic to fill 7 million DVDs. Side by side, they’d scale Mount Everest 95 times.
  3. The world’s 500,000+ data centers are large enough to fill 5,955 football fields.
  4. There are 30 billion pieces of content shared on Facebook and 230+ million tweets on Twitter, every day.
  5. There are nearly as many bits of information in the digital universe as there are stars in our actual universe.
  6. People wishing each other Happy New Year drove a 500% surge in smartphone data within just one year, according to 3UK whose customers used a whopping 80 terabytes (TB) on the 31st December 2011, compared to just 14 TBs on the same day in 2010.
  7. Google’s Eric Schmidt claims that every two days now we create as much information as we did from the dawn of civilization up until 2003.

Still looking skywards? This can go on for days, but you do get the gist: data is enormous and just lying out there with a huge potential to change the way we do business.

Why is it a Big deal?

Businesses are looking towards technology for coming up with new and profound ways to improve productivity and the internet has given them just that. Big data is useless unless there are methods to extract unknown patterns from the data and use them in remodeling the business activities. The competitive benefits of Big Data are likely to accrue to companies that can not only capture more and better data but also use that data effectively at scale.

“There is a big data revolution,” says Weatherhead University Professor Gary King. But it is not the quantity of data that is revolutionary. “The big data revolution is that now we can do something with the data.”

Big Data Process

“The importance of Big Data lies in improved statistical and computational methods, not in the exponential growth of storage or even computational capacity”, King explains. The doubling of computing power every 18 months (Moore’s Law) “is nothing compared to a big algorithm”—a set of rules that can be used to solve a problem a thousand times faster than conventional computational methods could. “A person, faced with a mountain of data, figured out that he would need a $2-million computer to analyze it. Instead, he asked Big Data Scientists to develop an algorithm that would do the same thing in 20 minutes—on a laptop: a simple example, but illustrative.”

What Big things can we do with it?

From Social Media to Medicine to Space, Big Data has changed the pace at which the fields of science and business grow.

The value created by Big Data in various fields:

Healthcare: The first full human genome sequence took five to 15 years to complete, and cost $1 billion to $3 billion. By 2009, eight years later, the cost had dropped to $100,000 and took a year. Now a genome sequence takes a little more than 24 hours and costs about $1,000—the point at which it can be paid for on a credit card. That simple statement alone underscores why the biomedical sciences have become so data-driven.

Google has analyzed clusters of search terms by region in the United States to predict flu outbreaks faster than was possible using hospital admission records.

Retail: In marketing, familiar uses of big data include “recommendation engines” like those used by companies such as Netflix and Amazon to make purchase suggestions based on the prior interests of one customer as compared to millions of others. Target famously (or infamously) used an algorithm to detect when women were pregnant by tracking purchases of items such as unscented lotions—and offered special discounts and coupons to those valuable patrons. Credit-card companies have found unusual associations in the course of mining data to evaluate people who buy anti-scuff pads for their furniture, for example, are highly likely to make their payments.

Government & Public: In the developed economies of Europe, government administrators could create more than €100bn ($123bn) in operational efficiency improvements alone by using Big Data – and that’s not including employing advanced analytic tools to reduce fraud and errors and boost the collection of tax revenues.

In the public realm, there are all kinds of applications: allocating police resources by predicting where and when crimes are most likely to occur; finding associations between air quality and health; or using genomic analysis to speed the breeding of crops like rice for drought resistance.

Space: Citizen Science projects such as Zooniverse, use the efforts and ability of volunteers to help scientists and researchers deal with the flood of data that confronts them. A pool of more than 1 million citizens from around the globe analyze data from the Kepler spacecraft to find alien planets around other stars.

Social Media: A recent application by TCS, known as SocialSoccer, is a unique smartphone app that brings you powerful insights and emotions on your favorite teams and players based on real-time Big Data analytics on social media during this FIFA World Cup.

India itself is a gigantic source of data, with 933 million mobile phone users (2nd highest in the world) and about 300 million internet users (3rd highest in the world). Imagine the data being produced by such big pool of data generators. Still at a nascent stage, Indian companies are hiking towards the development of big data algorithms and tools

The applications are as vast as the data itself, and in near future Big Data is going to be an essential competitive and evolution tool for all of mankind and technology alike. According to Gartner, Big data will drive $232 billion in spending through 2016, creating 4.4 million IT jobs globally to support Big Data.

• • •

Provided that businesses and governments understand the power of Big Data to deliver higher throughput, better value for consumers, and the immaculate growth in the global economy, there should be a strong enough incentive for them to act and overcome the barriers to its use. This will unleash possibilities of new competitiveness among companies, higher efficiency in the public sector that will enable better services, and enable firms and even whole economies to be more productive.

[About the Author: Ankit Purohit is a Marketing & Strategy Exec. SoftwareSuggest. Purohit is an IT Engineer and a PG in Information Systems Management & Marketing.]

Sources: Harvard Magazine Inc.| Ivey Business Journal

Leave a Reply