The Hortonworks Community Connection is now live. A completely rebuilt Q&A forum, Knowledge Base, Code Hub and more, backed by the experts in the industry.

You will be redirected here in 10 seconds. If your are not redirected, click here to visit the new site.

The legacy Hortonworks Forum is now closed. You can view a read-only version of the former site by clicking here. The site will be taken offline on January 31,2016

Pig Forum

Some pig statement fail at pig-0.12.0.2.0.6.0-76-withouthadoop.jar in HDP2.0

  • #43157
    Wang Wei
    Participant

    We use pig and hcatloader to read data in hive table with two partition: address,age
    It seems that it’s a pig parse issue,the loader statement can pass through the pig statement parser,but the filter can’t pass :
    B = FILTER A BY (address == ‘Shunyi’) AND (age == ’20’);

    After i degrade it to low version pig like pig-0.11.2.2.0.5.0-67-withouthadoop.jar,it’s ok,but we meet other issues.
    So
    If it’s a pig issue,when can we get the new version pig to use for hdp2.0?
    If not,how can we avoid it?

  • Author
    Replies
  • #44229
    abdelrahman
    Moderator

    Hi Wang,

    Is it possible for you to post the full pig script to test the use case?

    Thanks
    -Abdelrahman

    #50173
    Wang Wei
    Participant

    Hi Abdelrahman,
    Thanks for your response. The whole script:
    A = LOAD ‘default.customer’ USING org.apache.hcatalog.pig.HCatLoader();
    B = FILTER A BY (address == ‘Shunyi’) AND (age == ’20’);
    dump B;

    We also try Cloudera CDH5,the issue exist also.

    #50261
    Wang Wei
    Participant

    How to edit the comment?
    I want to delete the statement : We also try Cloudera CDH5,the issue exist also.

    #50270
    Wang Wei
    Participant

    We have know the reason:
    “register”(case-insensitive) is a key word in pig and the lastest pig(it’s ok for previous pig version) don’t allow the key word as schema’s column name, so if we use it as the schema’s column name,the exception will appear.

    B = FILTER A BY (address == ‘Shunyi’) AND (register == ’20′);

    The script above will make the issue,it’s a wrong usage.

    Thank abdelrahman for your response for my question.(I am not a good asker).

The forum ‘Pig’ is closed to new topics and replies.

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.