Simple MR job fails on HDP2.1 sandbox

to create new topics or reply. | New User Registration

This topic contains 0 replies, has 1 voice, and was last updated by  Param Dutta 11 months, 1 week ago.

  • Creator
  • #53742

    Param Dutta


    I am unable to get simple word_count to run on HDP2.1 sandbox. I get the exception attached at the bottom. On looking at the container logs, I see:
    ” Error: Could not find or load main class ”

    I verified that the classpath is correct. “yarn.application.classpath” in the job configuration looks like this:
    yarn.application.classpath = /etc/hadoop/conf,/usr/lib/hadoop/*,/usr/lib/hadoop/lib/*,/usr/lib/hadoop-hdfs/*,/usr/lib/hadoop-hdfs/lib/*,/usr/lib/hadoop-yarn/*,/usr/lib/hadoop-yarn/lib/*,/usr/lib/hadoop-mapreduce/*,/usr/lib/hadoop-mapreduce/lib/*

    I am not setting “yarn.application.classpath”, in my client application. It is automatically set and looks good to me. The class is in /usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-app-

    The stack in my client log:

    2014-05-15 10:28:15,121 INFO [Job] Job job_1400096640605_0002 failed with state FAILED due to: Application application_1400096640605_0002 failed 2 times due to AM Container for appattempt_1400096640605_0002_000002 exited with exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
    at org.apache.hadoop.util.Shell.runCommand(
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(
    at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(
    at java.util.concurrent.ThreadPoolExecutor.runWorker(
    at java.util.concurrent.ThreadPoolExecutor$

    Thanks for any help with this.


You must be to reply to this topic. | Create Account

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.