How to create HIVE table through PIG

This topic contains 0 replies, has 1 voice, and was last updated by  Pranav Singh 11 months, 1 week ago.

  • Creator
    Topic
  • #49224

    Pranav Singh
    Participant

    I have a use case where Pig UDF processes some data and generates either string or int etc. I need to store that back into a hive table.

    For this, I have to create Hive table in advance for that schema and I am using HCatStorer() to store the same in hive. How can I change my pig script to let that create the hive table for me on the fly.

    Please note that I would not easily know the schema in pig script, so cannot use HiveColumnarLoader as well.

    Example: Pig script generates output as id (int), terms (a,b,c – list of string) and some keyvalue pair. It needs to go in hive as int, array<string>, map<string,string>

    This may change in future, there can be more columns or different data types. How can I make this dynamic within the pig script.

You must be to reply to this topic. | Create Account

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.