Home Forums MapReduce Simple MR job fails on HDP2.1 sandbox

This topic contains 0 replies, has 1 voice, and was last updated by  Param Dutta 4 months ago.

  • Creator
    Topic
  • #53742

    Param Dutta
    Participant

    Hi,

    I am unable to get simple word_count to run on HDP2.1 sandbox. I get the exception attached at the bottom. On looking at the container logs, I see:
    ” Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster ”

    I verified that the classpath is correct. “yarn.application.classpath” in the job configuration looks like this:
    yarn.application.classpath = /etc/hadoop/conf,/usr/lib/hadoop/*,/usr/lib/hadoop/lib/*,/usr/lib/hadoop-hdfs/*,/usr/lib/hadoop-hdfs/lib/*,/usr/lib/hadoop-yarn/*,/usr/lib/hadoop-yarn/lib/*,/usr/lib/hadoop-mapreduce/*,/usr/lib/hadoop-mapreduce/lib/*

    I am not setting “yarn.application.classpath”, in my client application. It is automatically set and looks good to me. The class is in /usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app-2.4.0.2.1.1.0-385.jar.

    The stack in my client log:

    2014-05-15 10:28:15,121 INFO [Job] Job job_1400096640605_0002 failed with state FAILED due to: Application application_1400096640605_0002 failed 2 times due to AM Container for appattempt_1400096640605_0002_000002 exited with exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
    org.apache.hadoop.util.Shell$ExitCodeException:
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
    at org.apache.hadoop.util.Shell.run(Shell.java:418)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
    at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)

    Thanks for any help with this.

    Best,
    Param

You must be logged in to reply to this topic.