Help for YARN configuration
I have 6 nodes with 128GB of Ram, 32 cores on each, 9 disks and a VIC of 10Gb on each.
I would like to see the best performance of this hardware that I can have. For that I used DSFIO, Terasort, wordcount, ect.
I installed Hortonworks with ambari and the default configuration of Yarn and MRv2 are the following:
The rest are normally the default value (I think)
I found the result of my test really bad (terasort: 85minutes), can you give me some tips to configure yarn and MRv2 to have better performance?
I made some test with this configuration but the results are still not really good (terasort: 69minutes).
Thanks in advance,
Support from the Experts
A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.
Become HDP Certified
Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world