Manufacturing Does Hadoop

Use Apache Hadoop to Increase Production, Reduce Costs & Improve Quality

Manufacturing managers try to do three basic things: increase volume, reduce cost and improve quality. Without the right technology, these goals might seem to conflict. For example, how do we produce a better product while also reducing the cost to produce each unit?

Now relatively inexpensive sensors can gather and frequently transmit data along many points in the supply chain and production line. This flow of real-time sensor and machine data allows manufacturers to quickly identify problems as they occur. Our manufacturing customers also analyze historical data on HDP to move beyond reactive error avoidance to proactive process improvement.

Here is a general manufacturing reference architecture that reflects common system components and processes used by some of our manufacturing customers:

Manufacturing Ref Arch

Learn how manufacturers of electronics, aircraft, automobiles, chemicals, pharmaceuticals, household appliances and industrial equipment can use their modern manufacturing data architectures for the following use cases.

Assure Just-In-Time Delivery of Raw Materials

Manufacturers want to minimize the inventory that they keep on hand and prefer just-in-time delivery of raw materials. On the other hand, stock-outs can cause harmful production delays. Sensors and RFID tags reduce the cost of capturing supply chain data, but this creates a large, ongoing flow of data.

Hadoop can store this unstructured data at a relatively low cost. That means that manufacturers have more visibility into the history of their supply chains and they are able to see large patterns that might be invisible in only a few months of data. This intelligence can give manufacturers greater lead-time to adjust to supply chain disruptions. It also allows them to reduce supply chain costs and improve margins on the finished product.

Control Quality with Real-Time & Historical Assembly Line Data

High-tech manufacturers use sensors to capture data at critical steps in the manufacturing process. This data is useful at the time of manufacture, to detect problems while they are occurring. However, some subtle problems—the “unknown unknowns”—may not be detected at time of manufacture. Nevertheless, those may lead to higher rates of malfunction after the product is purchased.

When a product is returned with problems, the manufacturer can do forensic tests on the product and combine the forensic data with the original sensor data from when the product was manufactured. This added visibility, across a large number of products, helps the manufacturer improve the process and product to levels not possible in a data-scarce environment.

Avoid Stoppages with Proactive Equipment Maintenance

Today’s manufacturing workflows involve sophisticated machines coordinated across pre-defined, precise steps. One machine malfunction can stop the production line. Premature maintenance has a cost; there is an optimal schedule for maintenance and repairs: not too early, not too late.

Machine learning algorithms can compare maintenance events and machine data for each piece of equipment to its history of malfunctions. These algorithms can derive optimal maintenance schedules, based on real-time information and historical data. This can help maximize equipment utilization, minimize P&E expense, and avoid surprise work stoppages.

Increase Yields in Drug Manufacturing

Biopharma manufacturing requires careful monitoring and control of environment conditions. The goal of any production run is to maximize First Time Yield (FTY), which is a measure of the number of products that are made correctly the first time they come through the production line. Every percentage of increase in FTY represents a significant reduction in the costs of production.

FTY improvements are often blocked by poor visibility into operations. Sensors can provide raw data for improving that visibility, if the sensor data can be integrated with other existing data stores. A Hadoop data lake makes this integration easier, because Hadoop does not require an a priori schema prior to ingest. Also, Hadoop’s lower cost of storage means that a cluster can store more data, of more formats, for longer for discovering new relationships in the data.

Crowdsource Quality Assurance

Thoroughly tested products still have post-sale problems. Customers may not report problems to the manufacturer, but still complain about the product to their friends and family on social media. This social stream of data on product issues can augment product feedback from traditional support channels.

Hadoop stores huge volumes of social media sentiment data. Manufacturers can mine this data for early signals on how a product holds up throughout its lifecycle. This ability to learn about issues quickly and take early action to protect a product’s reputation is powerful for winning and maintaining customer loyalty.

See how WD, a Western Digital company, uses HDP to see their manufacturing data more quickly, save it for longer, and share it with more team members.


Try these Tutorials

Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.
Integrate with existing systems
Hortonworks maintains and works with an extensive partner ecosystem from broad enterprise platform vendors to specialized solutions and systems integrators.
Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.