Home Forums Oozie Classpath issues with oozie launcher job

This topic contains 6 replies, has 2 voices, and was last updated by  Yi Zhang 11 months, 2 weeks ago.

  • Creator
    Topic
  • #35889

    We use oozie to run our workflows on HDP 1.3. We installed the platform on a small test cluster using ambari.

    The problem that arises is when we submit the work flow job using oozie cli, the oozie launcher job’s task breaks with ClassNotFoundException. The class it cannot find is the SequenceFileOutputFormat from hadoop-core. I’m baffled about this because the classpath listing that oozie launcher task writes to stdout includes hadoop-core jar from HDP instalation and that jar exists and DOES have the class in question. I double checked the class name in work flow xml and it’s ok. I was running the same work flow successfully on by hand installed pseudo distributed cluster.

    Does anyone know what could be the problem?

Viewing 6 replies - 1 through 6 (of 6 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #42872

    Yi Zhang
    Moderator

    Hi Miljan,

    Thanks for sharing this knowledge! Wish there are some tools to check every hadoop property files…a typo can be disastrous. : (

    Yi

    Collapse
    #42754

    I didn’t copy-paste the xml, so I may have mistyped it or the post may have got reformatted (a post preview would be helpful here). Sorry for that. In short, if you have:
    class name\n
    inside value element then while parsing that xml the class name doesn’t get extracted correctly. When you remove the newline character, everything works fine.

    Collapse
    #42743

    Yi Zhang
    Moderator

    Hi Miljan,

    Thanks for sharing this knowledge! Somehow the newline doesn’t show up in the the forum post, but can xmllint detect it?

    –Yi

    Collapse
    #42652

    It’s all configured properly. I actually found out the reason for the problem and it’s rather interesting oozie/hadoop conf xml parsing bug/feature.

    The thing is, when you make a configuration xml and put something like

    <property>
    <name>mapreduce.map.class</name>
    <value>com.myapp.MyMapper</value>
    </property>

    it works properly, while if you do something like this (notice the newline in <value>)

    <property>
    <name>mapreduce.map.class</name>
    <value>com.myapp.MyMapper</value>
    </property>

    It doesn’t prune the newline from the mapper class name and so the class loader can’t find the class.
    This can happened because my eclipse IDE code formatter formatted the xml that way.

    Collapse
    #42642

    Yi Zhang
    Moderator

    Hi Miljan,

    How in your workflow.xml is the class defined? is oozie-site.xml pointing to the right hadoop conf location?
    oozie.service.HadoopAccessorService.hadoop.configurations

    Thanks,
    Yi

    Collapse
    #35890

    Here’s the stack trace and the part of the classpath listing:


    Classpath :
    ------------------------
    /etc/hadoop/conf
    /usr/jdk/jdk1.6.0_31/lib/tools.jar
    /usr/lib/hadoop/libexec/..
    /usr/lib/hadoop/libexec/../hadoop-core-1.2.0.1.3.2.0-111.jar
    /usr/lib/hadoop/libexec/../lib/ambari-log4j-1.2.5.17.jar
    etc...

    and the stack trace

    java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857)
    at org.apache.hadoop.mapreduce.JobContext.getOutputFormatClass(JobContext.java:235)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:970)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
    at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:910)
    at org.apache.oozie.action.hadoop.MapReduceMain.submitJob(MapReduceMain.java:97)
    at org.apache.oozie.action.hadoop.MapReduceMain.run(MapReduceMain.java:57)
    at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37)
    at org.apache.oozie.action.hadoop.MapReduceMain.main(MapReduceMain.java:40)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:491)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:365)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)

    Collapse
Viewing 6 replies - 1 through 6 (of 6 total)