Oozie Forum

Oozie and LZO problems

  • #31091
    Marc A.

    I am running HDP 1.2 and used Oozie successfully to run Java MR, Pig and Hive jobs.

    Then I installed Lzo support which now seems to break my Oozie.

    I have the same situation as documented here:

    Oozie can’t start any MR jobs and reports:
    ClassNotFound error (java.lang.ClassNotFoundException: Class com.hadoop.compression.lzo.LzoCodec not found)

    I tried several things that Oozie finds the library hadoop-lzo-0.5.0.jar:
    (1) cp /usr/lib/hadoop/lib/hadoop-lzo-0.5.0.jar /var/lib/oozie
    (2) cp /usr/lib/hadoop/lib/hadoop-lzo-0.5.0.jar /usr/lib/oozie/lib

    Can you help me how to fix this?

    Another question:
    Do you know what of purpose of /usr/lib/oozie/share/lib is?
    There are lots of jars in it.
    Additionally it is also in hdfs at /user/oozie/share/lib

    I appreciate any help.

to create new topics or reply. | New User Registration

  • Author
  • #31171
    Sasha J

    You should add LZO codec to oozie .war file
    Use ooze-setupsh or addtowar.sh script for this.
    Take a look to the Hortoworks docs:

    Thank you!

    Jayashankar VS

    As pointed out, here is the exact command from the documentation to add hadop-lzo.jar to ozzie.war. There is also one other thing with this command and the tricky part is you need to set the oozie PID in ozzie-env.sh

    export CATALINA_PID=/var/run/oozie/oozie.pid

    /usr/lib/oozie/bin/oozie-setup.sh -hadoop 0.20.200 /usr/lib/hadoop -extjs /usr/share/HDP-oozie/ext-2.2.zip -jars /usr/lib/hadoop/lib/hadoop-lzo-0.5.0.jar

    Executing the above command will bring the below output.

    /usr/lib/oozie/bin/oozie-setup.sh -hadoop 0.20.200 /usr/lib/hadoop -extjs /usr/share/HDP-oozie/ext-2.2.zip -jars /usr/lib/hadoop/lib/hadoop-lzo-0.5.0.jar
    setting OOZIE_CONFIG=/etc/oozie/conf
    setting OOZIE_DATA=/var/lib/oozie
    setting OOZIE_LOG=/var/log/oozie
    setting CATALINA_BASE=${OOZIE_DATA}/oozie-server
    setting CATALINA_TMPDIR=/var/tmp/oozie
    setting CATALINA_PID=/var/run/oozie/oozie.pid
    setting JAVA_HOME=/usr/java/default
    setting OOZIE_LOG=/var/log/oozie
    setting CATALINA_PID=/var/run/oozie/oozie.pid
    setting OOZIE_DATA=/var/db/oozie
    setting JAVA_LIBRARY_PATH=”/usr/lib/hadoop/lib/native/Linux-amd64-64″

    Injecting following Hadoop JARs


    New Oozie WAR file with added ‘Hadoop JARs, ExtJS library, JARs’ at /var/lib/oozie/oozie-server/webapps/oozie.war

    Marc A.

    Thanks for your support.

    Adding the library to the Oozie war file worked, as Sasha wrote.

    However, do you know, why it is not working when I set up the sharelib (that’s something I tried as well)

    I am wondering why I could not get it working with the sharelib…?
    Maybe one of you knows the reason for that?


    Jayashankar VS

    Not sure if I understand this question, the jar files shown below that you added were part of ozzie which is required for initializing oozie application.


    Can you please elaborate on this statement which says “However, do you know, why it is not working when I set up the sharelib (that’s something I tried as well)”?

    Marc A.

    Yes, let me elaborate on my previous statement.

    Without adding the library to Oozie war, I could start and stop Oozie but I could not see any errors when Oozie initializes.
    I could also execute the command “oozie admin -oozie http://sandbox:11000/oozie -status” but could not see any problems as well.

    Only when I started an Oozie job (no matter what kind of Oozie action in it), I got the ClassNotFound error message I mentioned.
    I was therefore thinking, that it is a “runtime problem” of my Oozie job which I could possibly solve when I add the lzo library to the Oozie sharedlib folders in HDFS – but that did not solve my problems.

    But after adding the lzo library to the Oozie war file, everything worked.


    Hi Marc,

    From command line please run:

    ps aux | grep -i oozie

    And post the output in this forum.


You must be to reply to this topic. | Create Account

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.