- 1.1 Big Data Basics
- 1.2 What Is Different?
- 1.3 What Does It Mean?
- 1.4 Transformations
- 1.5 Data-Driven Supply Chains
This chapter is from the book
This chapter is from the book
This chapter is from the book
1.2 What Is Different?
Companies have been capturing data for years and conducting analysis to gain market intelligence. It is natural to wonder what exactly is different today.
The difference is scale. This scale is in terms of the amount of data and the computing capability to analyze it. Combined, these elements can offer objective, evidence-based insights into virtually every aspect of a business. This has created a new level of competitiveness and the opportunity for companies to use this intelligence as a competitive advantage. We are living through a data explosion. According to a recent New York Times article, “Data is a vital raw material of the information economy, much as coal and iron ore were in the Industrial Revolution. But the business world is just beginning to learn how to process it all.”14
The three characteristics of big data—volume, velocity, and variety—are what make big data different from all the other data collected in the past (see Figure 1.2).15
)
Figure 1.2 The three Vs of big data
1.2.1 Volume
Today’s data is huge and data is everywhere. Consider that Google receives more than three billion queries every day, a volume that is thousands of times the quantity of all printed material in the U.S. Library of Congress; Facebook gets more than 10 million new photos uploaded every hour; Walmart conducts a million transactions per hour; and the New York Stock Exchange (NYSE) trades 490 million shares per day.16 The amount of data generated will continue to grow exponentially. In fact, the number of RFID tags sold globally is projected to rise from 12 million in 2011 to 209 billion in 2021.17
Manufacturers and retailers are collecting data all along their supply chains. This includes data from POS, GPS, and RFID data, to data emitted by equipment sensors, to social media feeds. Virtually all companies have information technology (IT) systems in their back offices. The world we live in is enveloped in data. When converting terabytes and exabytes into meaningful terms, it is estimated that the data that companies and individuals are producing and storing is equivalent to filling more than 60,000 U.S. Libraries of Congress.18 Where is this data going? It is accumulating in large pools growing larger by the minute. This is big data. In fact, many companies do not even recognize the data they possess and how valuable it is. Data can be traded or sold, and it has economic value. Data is the new asset.
1.2.2 Velocity
Yes, the data is large. The data is growing. And it is growing rapidly.
Data has become a deluge flowing into every aspect of business and everyday life. Companies are capturing exponentially growing volumes of transactional data. They are also capturing volumes of information about their customers, suppliers, and operations. Consider that there are millions of sensors embedded in physical devices all around us, including mobile phones, smart energy meters, automobiles, and industrial machines. These devices capture and communicate data in what is called the age of the Internet of Things.19
Companies and individuals are generating a tremendous amount of digital exhaust data. This is data that is created as a by-product of other activities and is generated just by going about everyday business. Consumers going about their day through texting, communicating, browsing, searching, and buying are creating digital trails. These trails can be captured, monitored, and analyzed. Social media sites, smartphones, PCs, and laptops have enabled billions of individuals around the world to continue adding to the amount of big data available. The growing amount of multimedia content has played a major role in the exponential growth in the amount of big data. In fact, each second of high-definition video, for example, generates more than 2,000 times as many bytes as required to store a single page of text.20
Companies are collecting data with increasingly greater granularity and frequency. They can capture every customer transaction. They can attach increasingly personal information to these transactions. As a result, they are collecting more information about consumer behavior in many different environments. Companies can then aggregate and disaggregate this data in infinite combinations to mathematically optimize and determine the best marketing strategies, custom-tailored shopping experiences, flexible and targeted product designs, and an optimized fulfillment system.
1.2.3 Variety
Big data is also in every form imaginable. Most of us think of data as numbers neatly stored in columns and rows. However, big data is in combinations of forms and this variety is growing. It is in the form of structured data that we are familiar with but may also be readings from sensors, GPS signals from cell phones, and POS and RFID data. It is also unstructured, such as text and voice messages, social networks feeds, and blogs.
Sources of big data are everywhere—from sensors such as RFID and POS data at retail checkouts, to geolocation information transmitted from GPS signals, to vibration and heat sensors on equipment, to social media feeds (think of Twitter “re-tweets” and Facebook “likes”), to maintenance logs, to customer complaints (see Figure 1.3). Most smartphones have GPS capabilities that enable tracking. Cars alone are stuffed with chips, sensors, and software that upload performance data to the carmakers’ computers when the vehicle is serviced. Typical mid-tier vehicles now have some 40 microprocessors and electronics that account for one third of their cost.21 How else does your car manufacturer inform you that it is time for service?
)
Figure 1.3 Sources of data are everywhere