Pig to load HCatalog table with ORC File storage

to create new topics or reply. | New User Registration

Tagged: , , ,

This topic contains 2 replies, has 2 voices, and was last updated by  Eugene Koifman 1 year, 2 months ago.

  • Creator
  • #54089

    I am trying to figure out a way to load Hive tables using Pig. I have the below questions:
    1. Do I need to create a HCatalog table for PIG to be able to load data or can it load directly to a Hive table
    2. How to create a Hcatalog table with ORC storage. I checked the documentation and HDP1.3 mentions HCatalog can support future Hive storage formats like ORC but i am not able to find references from HDP 2.1

Viewing 2 replies - 1 through 2 (of 2 total)

You must be to reply to this topic. | Create Account

  • Author
  • #54121

    Eugene Koifman

    Just create a Hive table and specify storage, then use HCatStorer to have Pig write it it.


    I have found a link which mentions that HCatalog works with ORC files but no confirmation anywhere else.

Viewing 2 replies - 1 through 2 (of 2 total)
Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.