Hortonworks Sandbox Forum

Is Tez installed in the Sandbox 2.0 ?

  • #43434

    The following page says that I can try tez using the sandbox
    http://hortonworks.com/hadoop/tez/
    But the last time I tried I couldn’t run tez , even after I installed tez, because of some protobuf error.
    Is tez installed in this version of the sandbox?

to create new topics or reply. | New User Registration

  • Author
    Replies
  • #43444
    Dave
    Moderator

    Hi Austin,

    No, Tez is not installed as part of the sandbox. You can configure the repo and install & configure it by following the documentation for the manual install.

    Thanks

    Dave

    #43615

    Hi Dave,
    I tried installing as per http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.0.0.2/bk_installing_manually_book/content/rpm-chap-tez.html
    but I get the following error. Any pointers as to where I might be going wrong?

    /usr/lib/tez/sbin/tez-daemon.sh start ampoolservice
    /usr/lib/tez/conf:/usr/lib/tez//*:/usr/lib/tez/lib/*:/etc/hadoop/conf:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/.//*:/usr/lib/hadoop-hdfs/./:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/.//*:/usr/lib/hadoop-yarn/lib/*:/usr/lib/hadoop-yarn/.//*:/usr/lib/hadoop-mapreduce/lib/*:/usr/lib/hadoop-mapreduce/.//*:/etc/hadoop/conf:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/.//*:/usr/lib/hadoop-hdfs/./:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/.//*:/usr/lib/hadoop-yarn/lib/*:/usr/lib/hadoop-yarn/.//*:/usr/lib/hadoop-mapreduce/lib/*:/usr/lib/hadoop-mapreduce/.//*::/usr/share/java/mysql-connector-java-5.1.17.jar:/usr/share/java/mysql-connector-java.jar:/usr/lib/hadoop-mapreduce/*:/usr/lib/tez/*:/usr/lib/tez/lib/*:/usr/share/java/mysql-connector-java-5.1.17.jar:/usr/share/java/mysql-connector-java.jar:/usr/lib/hadoop-mapreduce/*:/usr/lib/tez/*:/usr/lib/tez/lib/*
    starting ampoolservice, logging to /usr/lib/tez/logs/tez-root-ampoolservice-sandbox.hortonworks.com.out
    Exception in thread “main” java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/service/CompositeService
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
    at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
    Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.service.CompositeService
    at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
    … 12 more
    Could not find the main class: org.apache.tez.ampool.AMPoolService. Program will exit.

    #43616
    Dave
    Moderator

    Hi Austin,

    Can you check the following variables & properties are set:

    http://svn.apache.org/repos/asf/hadoop/common/trunk/hadoop-mapreduce-project/INSTALL

    Thanks

    Dave

    #43672

    Hi Dave,

    I checked and those properties are set.
    I ran the yarn hadoop examples and they run.
    I did a couple of echo statements in the startup script for tez “tez-daemon.sh” and the following are the exact statements that start the service. I still encounter the same error that I posted earlier.

    cd /usr/lib/tez
    nohup nice -n 0 /usr/jdk64/jdk1.6.0_31/bin/java -Xmx1024m -Dtez.log.dir=/usr/lib/tez/logs -Dtez.log.file=tez-root-ampoolservice-sandbox.hortonworks.com.log -Dtez.home.dir=/usr/lib/tez -Dtez.id.str=root -Dtez.root.logger=INFO,RFA org.apache.tez.ampool.AMPoolService –cli > /usr/lib/tez/logs/tez-root-ampoolservice-sandbox.hortonworks.com.out 2>&1 < /dev/null &

    THE CLASSPATH is : /etc/tez/conf:/usr/lib/tez//*:/usr/lib/tez/lib/*:/etc/hadoop/conf:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/.//*:/usr/lib/hadoop-hdfs/./:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/.//*:/usr/lib/hadoop-yarn/lib/*:/usr/lib/hadoop-yarn/.//*:/usr/lib/hadoop-mapreduce/lib/*:/usr/lib/hadoop-mapreduce/.//*:/etc/hadoop/conf:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/.//*:/usr/lib/hadoop-hdfs/./:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/.//*:/usr/lib/hadoop-yarn/lib/*:/usr/lib/hadoop-yarn/.//*:/usr/lib/hadoop-mapreduce/lib/*:/usr/lib/hadoop-mapreduce/.//*::/usr/share/java/mysql-connector-java-5.1.17.jar:/usr/share/java/mysql-connector-java.jar:/usr/lib/hadoop-mapreduce/*:/usr/lib/tez/*:/usr/lib/tez/lib/*:/usr/share/java/mysql-connector-java-5.1.17.jar:/usr/share/java/mysql-connector-java.jar:/usr/lib/hadoop-mapreduce/*:/usr/lib/tez/*:/usr/lib/tez/lib/*

    #43675
    Dave
    Moderator

    Hi Austin,

    The error you see is because it cannot find a library / jar in the CLASSPATH
    NoClassDefFoundError: org/apache/hadoop/yarn/service/CompositeService

    I would suggest you ensure that your classpath is set correctly and that the yarn libraries exist in the directories.

    Thanks

    Dave

    #43746

    Hi Dave,

    I got in touch with the Apache Tez user mailing list. It seems that the tez rpms in the hortonworks repo is old. Please let me know if someone has got tez working on the sandbox using “yum” – I couldn’t.
    I built Tez from source and it’s working now.
    For other’s who face this problem, you can read the mail chain here: http://mail-archives.apache.org/mod_mbox/incubator-tez-user/201311.mbox/browser

    Thanks,
    Austin

You must be to reply to this topic. | Create Account

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.