HDP on Windows – Other Forum

Map Reduce Job Performance on a multi node cluster

  • #50538
    SambaSivaRao Y


    We are facing performance issue while running a map reduce job on 4 node cluster compared to Single node.
    The job on multi node cluster is taking more time compared to Single node.

    Details of job’s run time :
    On a Single node : 26 min
    On a 4 node cluster : >35 min.

    Details of Single node :
    8 GB RAM

    Details of Multi node cluster :
    Master node : 8 GB RAM
    2 Data Nodes with 8GB RAM & 2 with 4 GB.

    We are also struggling to monitor the running job on multi node cluster.
    Please guide me on this.


to create new topics or reply. | New User Registration

  • Author
  • #52098
    SambaSivaRao Y

    Issue resolved, We are running the job from Eclipse instead of Command line.
    This is answer for another question at


    Hi SambaSivaRaoy ,
    I’am running a mapreduce job on 3 node cluster and im facing the same problem.
    The job on multinode cluster is taking more time compared to single node (35 min vs 51 min) , I noticed that the first 3 mappers run within seconds but the last one is taking minutes. Is it a problem of configuration??

    Master Node 3GB RAM
    Slave1 Node 3GB RAM
    Slave2 Node 4GB RAM

    SambaSivaRao Y


    Could you please provide me the details of job type (Whether you are loading data from DB or to DB ).


You must be to reply to this topic. | Create Account

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.