Oozie Forum

Classpath issues with oozie launcher job

  • #35889

    We use oozie to run our workflows on HDP 1.3. We installed the platform on a small test cluster using ambari.

    The problem that arises is when we submit the work flow job using oozie cli, the oozie launcher job’s task breaks with ClassNotFoundException. The class it cannot find is the SequenceFileOutputFormat from hadoop-core. I’m baffled about this because the classpath listing that oozie launcher task writes to stdout includes hadoop-core jar from HDP instalation and that jar exists and DOES have the class in question. I double checked the class name in work flow xml and it’s ok. I was running the same work flow successfully on by hand installed pseudo distributed cluster.

    Does anyone know what could be the problem?

to create new topics or reply. | New User Registration

  • Author
  • #35890

    Here’s the stack trace and the part of the classpath listing:

    Classpath :

    and the stack trace

    java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.lib.output.SequenceFileOutputFormat
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:857)
    at org.apache.hadoop.mapreduce.JobContext.getOutputFormatClass(JobContext.java:235)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:970)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
    at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:910)
    at org.apache.oozie.action.hadoop.MapReduceMain.submitJob(MapReduceMain.java:97)
    at org.apache.oozie.action.hadoop.MapReduceMain.run(MapReduceMain.java:57)
    at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37)
    at org.apache.oozie.action.hadoop.MapReduceMain.main(MapReduceMain.java:40)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:491)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:365)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)

    Yi Zhang

    Hi Miljan,

    How in your workflow.xml is the class defined? is oozie-site.xml pointing to the right hadoop conf location?



    It’s all configured properly. I actually found out the reason for the problem and it’s rather interesting oozie/hadoop conf xml parsing bug/feature.

    The thing is, when you make a configuration xml and put something like


    it works properly, while if you do something like this (notice the newline in <value>)


    It doesn’t prune the newline from the mapper class name and so the class loader can’t find the class.
    This can happened because my eclipse IDE code formatter formatted the xml that way.

    Yi Zhang

    Hi Miljan,

    Thanks for sharing this knowledge! Somehow the newline doesn’t show up in the the forum post, but can xmllint detect it?



    I didn’t copy-paste the xml, so I may have mistyped it or the post may have got reformatted (a post preview would be helpful here). Sorry for that. In short, if you have:
    class name\n
    inside value element then while parsing that xml the class name doesn’t get extracted correctly. When you remove the newline character, everything works fine.

    Yi Zhang

    Hi Miljan,

    Thanks for sharing this knowledge! Wish there are some tools to check every hadoop property files…a typo can be disastrous. : (


You must be to reply to this topic. | Create Account

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.