Western Digital does Hadoop

Optimizing Manufacturing with Longer Retention of Sensor Data

WD manufactures half of the world’s hard drives. With Hadoop and Hortonworks Data Platform, WD engineers see their manufacturing data more quickly, save it for longer, and share it with more team members. This means continuous improvement in its manufacturing process, which lowers costs and improves customer satisfaction.

Manufacturing Challenges Before Hortonworks Data Platform

The WD manufacturing process is state of the art and driven by data. The first phase of the
process occurs in the clean room, where each drive’s components are assembled. Sensors
capture data at each step, for each drive. Across about 200 million devices per year, this adds up to petabytes of information.

This sensor data is valuable and can be categorized in two ways. Some of that test data is
related to warranties on WD’s drives, and should be stored in the company’s traditional data
warehouse. The remainder of the data is not critical for warranty obligations; however, with
Apache Hadoop WD found they could store all of this information as opposed to partial subsets.

Before adding Apache Hadoop to its data architecture, the company was able to retain data
for between 3 months to 1 year. Access to that data was limited to a relatively small number of individuals within WD.

WD’s Data-Driven Solution

With Hadoop, WD continues to improve the quality of its products and manufacturing yields
(by reducing the number of scrapped drives at the end of the line.) The company chose
Hortonworks Data Platform (HDP) as the best solution to improve its data retention and data access.

WD had used a pure Apache Hadoop cluster and was pleased with initial results, but turned to
Hortonworks (and to the HDP distribution) to get more from the stable, reliable distribution and
so they could focus on their manufacturing process rather than the data platform.

The Hortonworks professional services team worked with WD to migrate its existing Hadoop
environment to HDP, and then to upgrade that cluster to include the most recent community
innovations found in the latest version of HDP. Throughout the process, WD could call on the
Hortonworks support team and draw on its deep experience supporting customers across many
use cases and industry verticals.

The Results: Superior Quality and Higher Yields

Hadoop and Hortonworks Data Platform delivered real business value for WD. According to Li Yi, Engineering Director at WD, “With Hadoop and HBase we can save a lot more test data for WD drives and can keep that data a lot longer. Hortonworks support gives us peace of mind, as we know we can depend on them to fix any operation issue quickly.”

Speed to Insight

Increased access to data has driven more internal demand for that data from across WD. The manufacturing team provides a Critical Parameter Dashboard that allows employees to drill down into the data. Hortonworks support gives us peace of mind, as we know we can depend on them to fix any operation issue quickly.Now with Hortonworks Data Platform, they can create that dashboard ten times more quickly than before. By satisfying the growing demand for data more quickly, the WD Hadoop team helps the entire organization make better decisions and respond more nimbly to changes in technology and the marketplace.

Longer Retention

Before Hadoop, WD retained data for between 3 months to one year. Hortonworks Data Platform extends the data retention time horizon to two years. With this additional data, WD engineers can identify subtler patterns in the data that may not surface over a shorter time horizon.

Better Access

WD employs about 87,000 individuals and many of those require a rich set of data to make decisions. More than 1,000 WD team members access the data stored on Hortonworks Data Platform. Those team members can use Apache HBase to search the data set and see results in less than one second.

About WD

WD, a Western Digital company, designs and makes storage devices that serve the primary markets for storage: enterprise and cloud data centers, consumer electronics, backup, the internet and other emerging markets such as automotive and home and small office networking.

WD’s highly sophisticated and data-rich manufacturing process produces 50-60 million devices every quarter and the company has honed its manufacturing processes since it was founded in 1970. Today, WD continues to innovate its manufacturing process, using Hadoop in combination
with its established data warehouse solutions to analyze huge amounts of manufacturing data.

  • 10x improvement in speed to insight
  • Doubling length of data retention
  • Searchable data for 1,000+ employees