Home Forums Pig Some pig statement fail at pig-0.12.0.2.0.6.0-76-withouthadoop.jar in HDP2.0

Tagged: 

This topic contains 4 replies, has 2 voices, and was last updated by  Wang Wei 4 weeks, 1 day ago.

  • Creator
    Topic
  • #43157

    Wang Wei
    Participant

    We use pig and hcatloader to read data in hive table with two partition: address,age
    It seems that it’s a pig parse issue,the loader statement can pass through the pig statement parser,but the filter can’t pass :
    B = FILTER A BY (address == ‘Shunyi’) AND (age == ’20′);

    After i degrade it to low version pig like pig-0.11.2.2.0.5.0-67-withouthadoop.jar,it’s ok,but we meet other issues.
    So
    If it’s a pig issue,when can we get the new version pig to use for hdp2.0?
    If not,how can we avoid it?

Viewing 4 replies - 1 through 4 (of 4 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #50270

    Wang Wei
    Participant

    We have know the reason:
    “register”(case-insensitive) is a key word in pig and the lastest pig(it’s ok for previous pig version) don’t allow the key word as schema’s column name, so if we use it as the schema’s column name,the exception will appear.

    B = FILTER A BY (address == ‘Shunyi’) AND (register == ’20′);

    The script above will make the issue,it’s a wrong usage.

    Thank abdelrahman for your response for my question.(I am not a good asker).

    Collapse
    #50261

    Wang Wei
    Participant

    How to edit the comment?
    I want to delete the statement : We also try Cloudera CDH5,the issue exist also.

    Collapse
    #50173

    Wang Wei
    Participant

    Hi Abdelrahman,
    Thanks for your response. The whole script:
    A = LOAD ‘default.customer’ USING org.apache.hcatalog.pig.HCatLoader();
    B = FILTER A BY (address == ‘Shunyi’) AND (age == ’20′);
    dump B;

    We also try Cloudera CDH5,the issue exist also.

    Collapse
    #44229

    abdelrahman
    Moderator

    Hi Wang,

    Is it possible for you to post the full pig script to test the use case?

    Thanks
    -Abdelrahman

    Collapse
Viewing 4 replies - 1 through 4 (of 4 total)