Hive / HCatalog Forum

Latest Hive and Tez installation

  • #34138
    Roshan Punnoose
    Participant

    Hi,

    With Hortonworks 2.0.4 I was able to get hive and tez running fine with the installation instructions from: http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.0.4.0/bk_installing_manually_book/content/rpm-chap-tez-5-4.html. However, after upgrading to the latest rpms, tez no longer starts. Every job becomes a Mapreduce job by default, instead of a tez job. Any ideas?

    I noticed that the HIVE_AUX_JARS_PATH no longer seems to support ‘:’, so I added this to my hive-env.sh script to load every jar from the tez installation:
    ###########################################
    TEZ_AUX_PATH=/etc/tez/conf
    # Folder containing extra ibraries required for hive compilation/execution can be controlled by:
    for f in /usr/lib/tez/*.jar; do
    TEZ_AUX_PATH=${TEZ_AUX_PATH},$f;
    done

    for f in /usr/lib/tez/lib/*.jar; do
    TEZ_AUX_PATH=${TEZ_AUX_PATH},$f;
    done

    export HIVE_AUX_JARS_PATH=”/usr/lib/hcatalog/share/hcatalog/hcatalog-core.jar,${TEZ_AUX_PATH}”
    #######################

    Also, updated the jars on hdfs.

    Roshan

to create new topics or reply. | New User Registration

  • Author
    Replies
  • #34157
    Robert
    Participant

    Hi Roshan,
    Just wondering if you made some tweaks prior to the latest updates? Have you tried just installing fresh and not via an upgrade?

    Regards,
    Robert

    #34237
    Roshan Punnoose
    Participant

    I started a new instance with a fresh install of hadoop/hive/tez (all the latest). Followed the steps above, and the hive queries still always default to using regular mapreduce.

    If I follow this tutorial: http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.0.0.2/bk_installing_manually_book/content/rpm-chap-tez-2.html. I get this exception: java.lang.ClassNotFoundException: org.apache.hadoop.yarn.api.records.DelegationToken

    Do I need to add yarn jars specifically to the hive/hadoop classpath?

You must be to reply to this topic. | Create Account

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.