The Hortonworks Community Connection is now live. A completely rebuilt Q&A forum, Knowledge Base, Code Hub and more, backed by the experts in the industry.

You will be redirected here in 10 seconds. If your are not redirected, click here to visit the new site.

The legacy Hortonworks Forum is now closed. You can view a read-only version of the former site by clicking here. The site will be taken offline on January 31,2016

Pig Forum

HBase Storage with HDP 2

  • #46444
    Aaron Zimmerman
    Participant

    When I try to load data from hbase using pig (via HDP 2.0), I get an error:
    Backend error : java.lang.ClassNotFoundException: Class org.apache.hadoop.hbase.mapreduce.TableSplit not found

    I cannot find this class as part of the latest hbase artifacts published by hortonworks, 0.96.0.2.0.6.0-76-hadoop2. Has this class been removed?

  • Author
    Replies
  • #46487

    Hi Aaron

    Have you done this setting in machine?
    1) COPY THESE FILES TO THE HADOOP LIBRARY.

    sudo cp /usr/lib/hive/lib/hive-common-0.7.0-cdh3u0.jar /usr/lib/hadoop/lib/
    sudo cp /usr/lib/hive/lib/hbase-0.90.1-cdh3u0.jar /usr/lib/hadoop/lib/
    sudo cp /usr/lib/hive/lib/hbase-0.90.1-cdh3u0.jar /usr/lib/hadoop/lib/

    2)CLOSE HBASE AND HADOOP USING FOLLOWING COMMOND
    /usr/lib/hadoop/bin/stop-all.sh
    /usr/lib/hbase/bin/stop-hbase.sh

    3) RESTART HBASE AND HADOOP USING COMMOND
    /usr/lib/hadoop/bin/start-all.sh
    /usr/lib/hadoop/bin/start-hbase.sh

    #46493
    Aaron Zimmerman
    Participant

    Kishore,
    Thanks for the response. I’ve not done that, are you referencing some documentation somewhere? Would I need to do this on all nodes in the cluster? I would have expected the classpaths and lib directories to be handled by Ambari, is this not the case?

    I dont see any hbase jars under /usr/lib/hive, but I do see a bunch under /usr/lib/hbase. But there doesn’t seem to be an “hbase-{version}” jar, just a bunch of hbase-{something} jars:

    hbase-client-0.96.0.2.0.6.0-76-hadoop2.jar
    hbase-common-0.96.0.2.0.6.0-76-hadoop2.jar

    etc.

    This is the same problem with a client application. Previoulsly, working with native apache hbase .94, there was an “hbase-0.94.x” maven artifact that I could add to the project, allowing me to create a subclass of Table Input Format. But it seems like this “hbase” artifact no longer exists, just a bunch of hbase-xxxxx libraries.

    #46550

    These all jar files are in your hive installation package.In your hive directory , go to the lib directory where you get all the .jar files.which you have to copy all that .jar files into hadoop/lib.

The forum ‘Pig’ is closed to new topics and replies.

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.