Home Forums Hue Pig script got stuck @ dump

This topic contains 5 replies, has 2 voices, and was last updated by  Stanley Nguyen 5 months, 2 weeks ago.

  • Creator
    Topic
  • #49831

    Stanley Nguyen
    Participant

    I think this is something to do when the pig script is executing from HUE. From the sandbox environment, it works fine but with multiple hosts configuration, it gets stuck @ dump statement. If I comment out “dump”, then it works. The same script executes flawlessly from GRUNT. Couldn’t really find any error from logs. I would appreciate if someone can point me to where to look.

    Thanks,

    Stan

Viewing 5 replies - 1 through 5 (of 5 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #49912

    Stanley Nguyen
    Participant

    Hi Dave,

    I made it work though I’m not sure why :). Fairly new with HDP. As I mentioned previously I have a 3 machines cluster but only one node manager is running on host #3. I added Node Manager to the other twos and it works now. Hopefully you or someone can explain the root cause. Technically it should have worked.

    Thanks

    Collapse
    #49868

    Stanley Nguyen
    Participant

    Hi Dave,

    These are the versions I got on my 3 machines setup:

    -bash-4.1$ rpm -qa | grep hue
    hue-2.3.0.2.0.6.0-102.el6.x86_64
    hue-server-2.3.0.2.0.6.0-102.el6.x86_64
    hue-beeswax-2.3.0.2.0.6.0-102.el6.x86_64
    hue-oozie-2.3.0.2.0.6.0-102.el6.x86_64
    hue-hcatalog-2.3.0.2.0.6.0-102.el6.x86_64
    hue-shell-2.3.0.2.0.6.0-102.el6.x86_64
    hue-pig-2.3.0.2.0.6.0-102.el6.x86_64
    hue-common-2.3.0.2.0.6.0-102.el6.x86_64

    On the sandbox VM that I downloaded, the script works fine and it got a newer version of HUE:
    [root@sandbox ~]# rpm -qa | grep -i hue
    hue-common-2.3.0.2.0.6.0-171.el6.x86_64
    hue-oozie-2.3.0.2.0.6.0-171.el6.x86_64
    hue-sandbox-1.2.1-174.noarch
    hue-tutorials-1.2.1-174.noarch
    hue-beeswax-2.3.0.2.0.6.0-171.el6.x86_64
    hue-pig-2.3.0.2.0.6.0-171.el6.x86_64
    hue-server-2.3.0.2.0.6.0-171.el6.x86_64
    hue-2.3.0.2.0.6.0-171.el6.x86_64
    hue-hcatalog-2.3.0.2.0.6.0-171.el6.x86_64
    hue-shell-2.3.0.2.0.6.0-171.el6.x86_64

    Collapse
    #49858

    Dave
    Moderator

    Hi Stan,

    What version of Hue are you running?
    rpm -qa | grep hue

    I’m having the same issue but in the grunt shell it works fine.

    Thanks

    Dave

    Collapse
    #49849

    Stanley Nguyen
    Participant

    Thanks Dave. I simply followed the tutorials (both for stock and baseball statistics). Anywhere this is the script:

    a = LOAD ‘stocks’ using org.apache.hcatalog.pig.HCatLoader();
    b = filter a by stock_symbol == ‘IBM’;
    c = group b all;
    d = foreach c generate AVG(b.stock_volume);
    dump d;

    From Job Browser, it stops @ 5%. Thanks

    Collapse
    #49842

    Dave
    Moderator

    Hi Stan,

    Could you paste your script here and I can run it against my cluster.
    How big is the data you are dumping?

    Thanks

    Dave

    Collapse
Viewing 5 replies - 1 through 5 (of 5 total)