Home Forums Hortonworks Sandbox Sandbox – Pig Basic Tutorial example is nbot working

This topic contains 40 replies, has 24 voices, and was last updated by  Don Estes 1 day, 21 hours ago.

  • Creator
    Topic
  • #17798

    Sankarg
    Member

    Hi, I just tried the following pig Basic Tutorial which is not working

    a = LOAD ‘nyse_stocks’ USING org.apache.hcatalog.pig.HCatLoader();
    b = FILTER a BY stock_symbol == ‘IBM’;
    c = group b all;
    d = FOREACH c GENERATE AVG(b.stock_volume);
    dump d;

    when i tried the syntax check, the following logs captured.

    013-03-17 14:35:28,456 [main] INFO org.apache.pig.Main – Apache Pig version 0.10.1.21 (rexported) compiled Jan 10 2013, 04:00:42
    2013-03-17 14:35:28,459 [main] INFO org.apache.pig.Main – Logging error messages to: /home/sandbox/hue/pig_1363556128447.log
    2013-03-17 14:35:41,945 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine – Connecting to hadoop file system at: file:///
    2013-03-17 14:35:45,555 [main] ERROR org.apache.pig.tools.grunt.Grunt – ERROR 1070: Could not resolve org.apache.hcatalog.pig.HCatLoader using imports: [, org.apache.pig.builtin., org.apache.pig.impl.builtin.]
    Details at logfile: /home/sandbox/hue/pig_1363556128447.log

    please do the needful to resolve this issue. Thank you!

    Regards,
    Sankar

Viewing 10 replies - 31 through 40 (of 40 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #18836

    Larry Liu
    Moderator

    Hi, Kirill,

    Just wondering if you are using the new version of sandbox? Have you been able to try what peter suggested?

    sed -i ’49s/.*/includeHCatalog=true;/’ /usr/lib/pig/bin/pig

    Larry

    Collapse
    #18800

    Kirill Wood
    Member

    I tried this basic pig example too and got the same error as Sankarg:

    a = LOAD ‘nyse_stocks’ USING org.apache.hcatalog.pig.HCatLoader();
    b = FILTER a BY stock_symbol == ‘IBM’;
    c = GROUP b all;
    d = FOREACH c GENERATE avg(b.stock_volume);
    dump d;

    Syntax check gave me this error:
    ERROR 1070: Could not resolve org.apache.hcatalog.pig.HCatLoader using imports: [, org.apache.pig.builtin., org.apache.pig.impl.builtin.]

    As Larry suggested, I edited the last line of the file /usr/bin/pig to exec /usr/lib/pig/bin/pig -useHCatalog “$@”.

    The previous error is gone but now it throws another error:
    2013-03-23 20:46:30,499 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine – Connecting to hadoop file system at: file:///
    2013-03-23 20:46:32,452 [main] WARN org.apache.hadoop.hive.conf.HiveConf – DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.
    2013-03-23 20:46:32,739 [main] INFO hive.metastore – Trying to connect to metastore with URI thrift://sandbox:9083
    2013-03-23 20:46:33,160 [main] INFO hive.metastore – Waiting 1 seconds before next connection attempt.
    2013-03-23 20:46:34,160 [main] INFO hive.metastore – Connected to metastore.
    2013-03-23 20:46:36,073 [main] WARN org.apache.hadoop.hive.conf.HiveConf – DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.
    2013-03-23 20:46:39,105 [main] ERROR org.apache.pig.tools.grunt.Grunt – ERROR 1070: Could not resolve avg using imports: [, org.apache.pig.builtin., org.apache.pig.impl..builtin.]
    Details at logfile: /home/sandbox/hue/pig_1364096789805.log

    log file:
    Failed to parse: Pig script failed to parse:
    Failed to generate logical plan. Nested exception: java.lang.RuntimeException: Cannot instantiate: avg at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:184)

    How can I fix this?

    Thank you

    Collapse
    #18774

    Hi Sankarg, you could fix it by running next line on VM:
    sed -i '49s/.*/includeHCatalog=true;/' /usr/lib/pig/bin/pig
    or use an instruction, that Larry suggested

    Collapse
    #18625

    Seth Lyubich
    Keymaster

    Hi Rahul,

    Can you please clarify which eror you are referring too?

    Thanks,
    Seth

    Collapse
    #18605

    Rahul Jolly
    Member

    Hi Ted – what would be the fix for this error we’ve been seeing below. I am getting the same.
    Would you please let us know the steps we can take to resolve this.

    Thank you.
    Rahul

    Collapse
    #18168

    Larry Liu
    Moderator

    Hi, Sankar,

    Here is a workaround you could try to see if it works:

    1. Login to sandbox as root.
    2. Edit file /usr/bin/pig
    3. Replace the last line with the following line:
    exec /usr/lib/pig/bin/pig -useHCatalog “$@”

    I will continue looking into other solution.

    Thanks
    Larry

    Collapse
    #18104

    Larry Liu
    Moderator

    Hi, Sankar,

    Can you please provide the log from your error message: /home/sandbox/hue/pig_1363556128447.log

    From my initial look, the error said “2013-03-17 14:35:45,555 [main] ERROR org.apache.pig.tools.grunt.Grunt – ERROR 1070: Could not resolve org.apache.hcatalog.pig.HCatLoader using imports: [, org.apache.pig.builtin., org.apache.pig.impl.builtin.]“. It is mostly likely the HCatLoader is not in pig classpath.

    Larry

    Collapse
    #17959

    Sankarg
    Member

    I removed all the lines in the pig code example except the very first line and tried to run the syntax check and i could noticed the message that Detected Tx hang in the virtual box.

    Collapse
    #17902

    tedr
    Member

    Hi Sankar,

    I could not replicate your problem. I have noticed that sometimes VirtualBox can import the VM incorrectly, you could try re-importing the vm, while I continue to try and replicate your issue.

    Thanks,
    Ted.

    Collapse
    #17884

    tedr
    Moderator

    Hi Sankar,

    Thanks for trying the Hortonworks Sandbox,

    I am looking into why this might be happening and what to do about it. I will get back to you as soon as I have something definitive.

    Thanks,
    Ted.

    Collapse
Viewing 10 replies - 31 through 40 (of 40 total)