Notes

The Big Data Phenomenon (In the World)

The Big Data Phenomenon (In Italy)

"Traditional" Data Sources Operational sources referring companies daily operational activity:

• Software for production management
• Software for purchasing management
• Software for orders and deliveries management
• Software for accounting
• Software for personnel management
• Software for customer management
• Software for back-office management
• Software for financial instruments management and measurement

"Emergent" Data Sources Next to the data produced by management software's there might be systems closer to the production that generate huge amounts of data:

• DCS (Distributed Control System) computer systems for the control of industrial installations. Generate data on the state of the installations through sensors connected to the components running measurements at very small intervals.
• Data generated by scientific equipment for measurement and analysis
• Data generated by medical and diagnostic equipment
• High Frequency Trading Systems
• Web 2.0: Blog Posts, Tweets, Facebook comments & likes, images & videos
• IOT: Internet Of Things

Type of Data

"Classical" definition of Big Data Data with three characteristics:

• Volume (Large amounts of data)
• Variety (Variety of structures, data types, sources, Structural Complexity and Unstructured or semi-structured data)
• Velocity (The speed with which they are produced

The other 2 V's of the classic definition

• Value (It expresses a result of the use of big data: the ability to generate value)
• Veracity (It expresses critical issues: not everything that is on the internet or in the corporate data assets is trustworthy!)

Estimations There are certain estimates that see by 2025, the set of all the data digitally created and consumed in a year (books, videos, music, etc.), it will be equal to 180 Zettabyte. Other estimates say 160 Zettabyte for 2025 (IDC).

But do we have to analyze them all? Let's clarify The name "Big" Data, the traditional definition and some publications, are misleading. The Big Data is not only the data with large volumes. They come from the Web, but also from other sources (both new and traditional). The impressive numbers which are found in the statistics they refer to the data created throughout the world, we hardly need to use them all...

Big Data Definition Big Data is a new concept of potential Business knowledge that leverages on today variety of data formats and sources, on improved data velocity and, thus, on today increasing data volume, to generate new insights previously considered beyond our capability and to find new business value by using proper enablers.

Big Data Framework

An alternative definition (more technical) The Big Data are: Data that cannot be analyzed on a single machine or that is not convenient to analyze through traditional technologies (hw or sw). Can then be: Data with such volumes that the analysis on RDBMS is impossible or very costly Unstructured data difficult to keep in an RDBMS. Data that may need sophisticated analytical techniques to extract value, although this is not always true. Data that arises both structured and unstructured (or semi-structured). However: unstructured data (or semi-structured) must be processed and placed in the form of structured data, before it can be analyzed.

IOT (Internet Of Things) Network of objects containing electronic parts, software, sensors and connectivity. They add value to the product or service: exchange data with the producer and with other connected devices. Interact through the Internet.

IOT & Smart Cities

Big Data Phenomenon: Triggers

Analysis Process

Example of Big Data Applications in Business

• Reduce Time To Market
  • Introducing new products or services involves many life cycle stages, some of which are easier to accelerate than others; for the past couple of decades, drug manufacturers have been using clinical trial simulations to speed learning, reduce costs, and limit unnecessary burdens on patients participating in the trials.
  • Armed with the power of the cloud and big data, the clinical trial simulations can be done faster in ways that benefit the manufacturer and patients.
  • Bristol-Myers Squibb reduced the time it takes to run clinical trial simulations by 98% by extending its internally hosted grid environment into the AWS Cloud; the company has also been able to optimize dosing levels, make drugs safer, and require fewer blood samples from clinical trial patients.
  • As a result of the move, Bristol-Myers Squibb was able to reduce the number of clinical trial subjects in a pediatric study from 60 to 40, while shortening the length of the study by more than a year.
• Optimize The Workforce
  • Some HR departments are using talent analytics and big data to reduce costs and effectively manage workforce-related issues.
  • The data allows them to select new hires that are a better fit for the company, reduce employee turnover, understand the skills and output of the existing workforce, and determine the talent life the company needs moving forward.
  • Xerox used big data to reduce the attrition rate in its call centers by 20%: to do that, it had to understand what was causing the turnover, and determine ways to improve employee engagement.
• Improve Financial Performance
  • Corporate finance departments are moving beyond periodic reporting and BI, using big data to reduce risks and costs, identify opportunities, and improve the accuracy of forecasts: specifically, they're using data to identify risky customers, monitor suppliers, thwart fraud, pinpoint revenue leaks, and inform new or more efficient business models.
  • A recent partnership between The Weather Company and IBM will allow companies to better manage the impact of half a trillion dollars annually in the US alone.
  • The weather data is being collected from more than 100,000 weather sensors and aircraft, as well as millions of smartphones, buildings, and moving vehicles; that data is combined with data from other sources to yield 2.2 billion unique forecast points, and an average of more than 10 billion forecasts on an active weather day.
  • Retailers will be able to use the data to adjust staffing and supply chain strategies. Energy companies will be able to improve supply and demand forecasting. Insurance companies will be able to warn policy holders of severe weather conditions, so they can minimize the possibility of car damage in the event of a hail storm, for example.
• Sell Intelligently
  • Slight modifications to sales and marketing strategies can have a profound effect on the bottom line, especially when informed by big data.
  • Imagine a direct mail campaign with a coupon return rate of more 70% within six weeks of the mailing: according to the Direct Marketing Association, the average direct mail return rate is 3.7%.
  • How does grocery store chain Kroger do it? For one thing, it personalizes its direct mailer based on the shopping history of the individual customer; Kroger also has a loyalty card program that is rated No. 1 in the Grocery industry.
  • More than 90% of its customers use its loyalty card wen they purchase products; although there are many factors that have collectively enabled Kroger'S financial performance, at least part of its continued worth over 45 consecutive quarters has been attributed to its customer loyalty programs.
• Minimize Equipment And Asset Failures
  • Businesses want to avoid unnecessary disruption and customer angst.
  • Now that sensors are being embedded into just about everything, companies are using the data to determine when maintenance is required for planes, trains, automobiles, and even household appliances.
  • Ideally, when an issue has arisen, companies want to understand the issue, what caused it, and how it can be resolved, preferably before a maintenance professional or crew is dispatched.
  • Pratt&Whitney, a unit of United Technologies Corp., is attempting to reduce unplanned aircraft engine maintenance. According to AirInsight.com, today's engines collect about 100 parameters in multiple snapshots while a plane is in flight. By comparison, a new-generation engine is able to collect about 5,000 parameters continuously in flight. The process generates about 2 petabytes of data. Using the data, Pratt&Whitney and its partner IBM are trying to enable proactive maintenance.
• Leverage Customer Lifetime Value
  • Today's empowered customers are more demanding and fickle than ever: maintaining or increasing market share requires businesses to understand as much as possible about their customers, continually improve their products and services, and be willing to adapt their business models to reflect the actual needs of their customers.
  • Avis Budget has committed to doing all of this. It implemented an integrated strategy to increase market share, which has yielded hundreds of millions of dollars in additional revenue: the initiative involved determining the value of customers, segmenting them, and offering tiered incentives to improve customer loyalty.
  • To do this, its IT partner CSC applied a model that predicts lifetime value to Avis Budget's customer database, and then validated it using a multichannel marketing campaign and accompanying analytics; the customer valuation data is now combined with other data, including rental history, service issues, demographics, corporate affiliation, and customer feedback.
  • Avis Budget is also collecting and analyzing social media data: it has a team of social media specialists who respond to brand mentions.
  • The company recently updated its website to further improve customer experiences, and it is using big data to forecast regional demand for fleet placements and pricing.