Home Forums HDP on Windows – Non Installation issues Map Reduce Job Performance on a multi node cluster

This topic contains 3 replies, has 2 voices, and was last updated by  SambaSivaRao Y 5 months ago.

  • Creator
    Topic
  • #50538

    SambaSivaRao Y
    Participant

    Hi,

    We are facing performance issue while running a map reduce job on 4 node cluster compared to Single node.
    The job on multi node cluster is taking more time compared to Single node.

    Details of job’s run time :
    On a Single node : 26 min
    On a 4 node cluster : >35 min.

    Details of Single node :
    8 GB RAM

    Details of Multi node cluster :
    Master node : 8 GB RAM
    2 Data Nodes with 8GB RAM & 2 with 4 GB.

    We are also struggling to monitor the running job on multi node cluster.
    Please guide me on this.

    Thanks,
    SambaShiva.

Viewing 3 replies - 1 through 3 (of 3 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #56163

    SambaSivaRao Y
    Participant

    Hi,

    Could you please provide me the details of job type (Whether you are loading data from DB or to DB ).

    Thanks,
    SambaSiva.

    Collapse
    #53727

    IngEnsi
    Participant

    Hi SambaSivaRaoy ,
    I’am running a mapreduce job on 3 node cluster and im facing the same problem.
    The job on multinode cluster is taking more time compared to single node (35 min vs 51 min) , I noticed that the first 3 mappers run within seconds but the last one is taking minutes. Is it a problem of configuration??

    Details:
    Master Node 3GB RAM
    Slave1 Node 3GB RAM
    Slave2 Node 4GB RAM

    Collapse
    #52098

    SambaSivaRao Y
    Participant

    Issue resolved, We are running the job from Eclipse instead of Command line.
    This is answer for another question at

    http://hortonworks.com/community/forums/topic/map-reduce-job-run-from-eclipse/

    Collapse
Viewing 3 replies - 1 through 3 (of 3 total)