Hadoop is Transforming Manufacturing

Use Apache Hadoop to Increase Production, Reduce Costs & Improve Quality

Manufacturing managers try to do three basic things: increase volume, reduce cost and improve quality. Without the right technology, these goals might seem to conflict. For example, how do we produce a better product while also reducing the cost to produce each unit?

Now relatively inexpensive sensors can gather and frequently transmit data along many points in the supply chain and production line. This flow of real-time sensor and machine data allows manufacturers to quickly identify problems as they occur. Our manufacturing customers also analyze historical data on HDP to move beyond reactive error avoidance to proactive process improvement.

Here is a general manufacturing reference architecture that reflects common system components and processes used by some of our manufacturing customers:

Manufacturing Ref Arch

Learn how manufacturers of electronics, aircraft, automobiles, chemicals, pharmaceuticals, household appliances and industrial equipment can use their modern manufacturing data architectures for the following use cases.

Assure Just-In-Time Delivery of Raw Materials

Manufacturers want to minimize the inventory that they keep on hand and prefer just-in-time delivery of raw materials. On the other hand, stock-outs can cause harmful production delays. Sensors and RFID tags reduce the cost of capturing supply chain data, but this creates a large, ongoing flow of data.

Hadoop can store this unstructured data at a relatively low cost. That means that manufacturers have more visibility into the history of their supply chains and they are able to see large patterns that might be invisible in only a few months of data. This intelligence can give manufacturers greater lead-time to adjust to supply chain disruptions. It also allows them to reduce supply chain costs and improve margins on the finished product.

Control Quality with Real-Time & Historical Assembly Line Data

High-tech manufacturers use sensors to capture data at critical steps in the manufacturing process. This data is useful at the time of manufacture, to detect problems while they are occurring. However, some subtle problems—the “unknown unknowns”—may not be detected at time of manufacture. Nevertheless, those may lead to higher rates of malfunction after the product is purchased.

When a product is returned with problems, the manufacturer can do forensic tests on the product and combine the forensic data with the original sensor data from when the product was manufactured. This added visibility, across a large number of products, helps the manufacturer improve the process and product to levels not possible in a data-scarce environment.

Avoid Stoppages with Proactive Equipment Maintenance

Today’s manufacturing workflows involve sophisticated machines coordinated across pre-defined, precise steps. One machine malfunction can stop the production line. Premature maintenance has a cost; there is an optimal schedule for maintenance and repairs: not too early, not too late.

Machine learning algorithms can compare maintenance events and machine data for each piece of equipment to its history of malfunctions. These algorithms can derive optimal maintenance schedules, based on real-time information and historical data. This can help maximize equipment utilization, minimize P&E expense, and avoid surprise work stoppages.

Increase Yields in Drug Manufacturing

Biopharma manufacturing requires careful monitoring and control of environment conditions. The goal of any production run is to maximize First Time Yield (FTY), which is a measure of the number of products that are made correctly the first time they come through the production line. Every percentage of increase in FTY represents a significant reduction in the costs of production.

FTY improvements are often blocked by poor visibility into operations. Sensors can provide raw data for improving that visibility, if the sensor data can be integrated with other existing data stores. A Hadoop data lake makes this integration easier, because Hadoop does not require an a priori schema prior to ingest. Also, Hadoop’s lower cost of storage means that a cluster can store more data, of more formats, for longer for discovering new relationships in the data.

Read about how Merck Research Laboratories optimized pharmaceutical manufacturing with Hortonworks Data Platform.

Crowdsource Quality Assurance

Thoroughly tested products still have post-sale problems. Customers may not report problems to the manufacturer, but still complain about the product to their friends and family on social media. This social stream of data on product issues can augment product feedback from traditional support channels.

Hadoop stores huge volumes of social media sentiment data. Manufacturers can mine this data for early signals on how a product holds up throughout its lifecycle. This ability to learn about issues quickly and take early action to protect a product’s reputation is powerful for winning and maintaining customer loyalty.

FireKing Built a 360-Degree View of Its Field Operations

Many manufactured products have a lifecycle that depends on regular servicing and upkeep by field technicians. Hortonworks customer FireKing uses HDP to overcome challenges around processing its raw operational data on how its field technicians service the company’s safes, cash-management systems and commercial locking hardware.

Compilation and analysis of that field operations data used to be time-consuming, tedious, and error-prone. Now FireKing uses HDP to process and analyze structured data such as work order and payroll records, as well as unstructured geo-location data on service locations. By combining data from multiple sources originating from both the Field Operations and HR departments, the company can now understand the total volume of work performed each day and combine it with granular insight into past service demand. This new insight means that FireKing can confidently predict future service needs and then optimize staffing to match that demand.

See how WD, a Western Digital company, uses HDP to see their manufacturing data more quickly, save it for longer, and share it with more team members.


Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.