The Hortonworks Community Connection is now live. A completely rebuilt Q&A forum, Knowledge Base, Code Hub and more, backed by the experts in the industry.

You will be redirected here in 10 seconds. If your are not redirected, click here to visit the new site.

The legacy Hortonworks Forum is now closed. You can view a read-only version of the former site by clicking here. The site will be taken offline on January 31,2016

HBase Forum

How To Create Partitions In Hbase Table As Like Hive Table Partitions

  • #37769
    Hari Krishna

    We are planning to migrate from CDH3 to CDH4, as part of this migration we also planning to bring HBASE into out system because it also updates to the data, in CDH3 we are using Hive as warehouse.

    Here we are having the major problem in migration, Hive supports partitions to tables. And our system has many tables in different schemas and some tables has partitions base on date, we have the history of data from last 5 years (365 * 5 partitions exists in some tables).

    We want to achieve the same behavior in HBase also, when I browsed I couldnt find the solution for creating partitions in HBase. Can any one help me in implementing this partition wised table creation in HBase.

    Its really great help if we can find at least some work around solutions also.

  • Author
  • #38241

    Hi Har,

    If I may suggest using the Hive/HBase integration. From HBase table you can take its snapshot and populate it into Hive partitioned table using HBase/Hive integration.
    Here is an example:
    1. Create HBase table named ‘test’ from HBase shell:

    create ‘test’,’mstore’

    2. Create two Hive tables. The first table named ‘user’ is in Hive and is where the data is stored in Hive. The second table named ‘user_insert’ maps Hive table to the HBase table ‘test’ that is created in step 1.

    name string,
    id bigint,
    recipient_id bigint,
    type int
    lines terminated by ‘\n’
    LOCATION ‘/user/hive/user’;
    — The table will need to be partitioned by dt

    CREATE EXTERNAL TABLE user_insert (
    name string,
    id bigint,
    recipient_id bigint,
    type int
    STORED BY ‘org.apache.hadoop.hive.hbase.HBaseStorageHandler’
    WITH SERDEPROPERTIES (“hbase.columns.mapping” = “:key,mstore:name,mstore:id,mstore:recipient_id,mstore:type”)
    TBLPROPERTIES (“” = “test”);

    3. Insert data from table user to table user_insert. This will populate HBase table named ‘test’.

    set hbase.zookeeper.quorum=zk1,zk2,zk3 ;
    set hive.zookeeper.client.port=2181;
    set zookeeper.znode.parent=/hbase-unsecure ;

    — The table will need to be partitioned by dt

    In order to have all the required jars without the use having hive do “add jar”, please follow these steps:
    – Create a directory called auxlib
    #mkdir /usr/lib/hive/auxlib

    – Copy all of the following Jars to the previous directory:
    /usr/lib/zookeeper/zooNkeeper- ;


The forum ‘HBase’ is closed to new topics and replies.

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.