Absolutely! Hadoop cluster in a sense is a supercomputer where multiple nodes of cluster work together to achieve a common goal. That is how the MapReduce, Tez works and that’s how machine learning algorithms are executed using libraries like Mahout and R on hadoop.
Quite a few weather researchers are already using Hadoop for this purpose.
WRF with Hadoop
Is it possible build Weather Research and Forecasting (WRF) model in hadoop? Currently we are using computer cluster where there is one head node and 10 compute node; when a job submitted in the system through the head node, other 10 compute node perform the same submitted job. the total system act like a super-computer where total combined processing power used to perform the same job. is this possible with HDP?
Support from the Experts
A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.
Become HDP Certified
Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world