Home Forums Hortonworks Sandbox Unreliable

This topic contains 13 replies, has 3 voices, and was last updated by  Michael Hawk 1 week, 4 days ago.

  • Creator
    Topic
  • #56793

    Michael Hawk
    Participant

    I’ve spent 4 days off and on trying to get the tutorial 1 to work in sandbox. Logged onto the forums and found that others had similar problems to mine and their problems were not fixed. I tried a couple of things suggested like a to define HCAT_HOME. Nothing has worked. I still get a note that it is looking for a library.

    ls: cannot access /usr/lib/hive/lib/slf4j-api-*.jar: No such file or directory

    Could anyone provide some enlightenment?

Viewing 13 replies - 1 through 13 (of 13 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #57081

    Michael Hawk
    Participant

    Enabled ambari. Started and stopped yarn. Launched a one line job: batting = LOAD ‘Batting.csv’ USING PigStorage(‘,’); Almost immediately got the following error:
    WebHdfsException at /pig/start_job/
    IOException: File /tmp/.pigjobs/hue/runs_1405002731/script.pig could only be replicated to 0 nodes instead of minReplication (=1). There are 1 datanode(s) running and no node(s) are excluded in this operation.
    at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1430)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2696)
    at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:584)
    at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:440)
    at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)
    (error 403)
    pig is still running. nothing registered in Job Browser

    Collapse
    #57080

    iandr413
    Moderator

    Try going to http://127.0.0.1:8000/about/ and enabling Ambari, It will be helpful in working through the excercises.

    Collapse
    #57079

    Michael Hawk
    Participant

    I went to http://127.0.0.1/8080. Got a msg saying this page is not available. Am using chrome.

    Collapse
    #57078

    iandr413
    Moderator

    Hi Michael,
    Go to http://127.0.0.1:8080/ and login as admin/admin. From there you can see running services and manage those services.

    Collapse
    #57077

    Michael Hawk
    Participant

    How do I restart YARN using AMBARI?

    Collapse
    #57074

    iandr413
    Moderator

    Michael and Jon,
    I looked at the tutorial at http://hortonworks.com/hadoop-tutorial/hello-world-an-introduction-to-hadoop-hcatalog-hive-and-pig/ and the cases and quotes look correct. Was this a typo on your end or do you see this somewhere incorrectly on our site?

    If your job does not start Michael, you can try to restart YARN via Ambari. This will clear out any jobs that are stuck that could be preventing further runs.

    Ian

    Collapse
    #57038

    Michael Hawk
    Participant

    I spent 5 days trying to get the example from tutorial 1 to work. Finally did. Then start tutorial 2. Tried only first line of example code. Failed. Actually never got started. Job Browser said it was accepted. Dropped back and tried to run the example from tutorial1, the one I had working the other day. It now fails too, or rather doesn’t start.

    What now?

    Collapse
    #57026

    Michael Hawk
    Participant

    I have moved onto Tutorial2. Went thru the lesson. This time I was smarter with the exercise. Looked at the Forums to find a version of the code someone found successful. I only did one line of code to start off with. Did a cut and paste. Made sure the single quotes were properly entered. Made sure the words LOAD and USING were uppercase. Hit the execute button. Job just hung. history said it was running. Job Browser said the job was “Accepted”. Went back to the pig screen. No evidence of the job. Went to History screen. Deleted the job. Went to Job Browser. It still said job was accepted.

    Any ideas?

    Collapse
    #56893

    Michael Hawk
    Participant

    I am using sandbox 2.1. The jar file in the error msg is not in the directory cited. Sometimes the job fails, sometimes it runs “successfully” but the output file has 0 bytes.
    Today I went back and traced my steps to gather evidence, capture logs, etc. I copied the code from Joe Petitta. Still got error msgs. Then typed in the single quotes in both line 1 and 2. Script worked.
    Jar file is still missing. but script works.

    Collapse
    #56852

    Jon Petitta
    Participant

    I figured it out, you need to look deeper into the exception stack to find the true error.

    Mine turned out to be because of a case issue for a function call.

    Even though auto complete for the form allows all lower for the function ‘avg’, the interpreter only likes all caps ‘AVG’.

    This script works.

    a = LOAD ‘default.nyse_stocks’ USING org.apache.hcatalog.pig.HCatLoader();
    b = filter a by stock_symbol == ‘IBM’;
    c = group b all;
    d = foreach c generate AVG(b.stock_volume);
    dump d;

    This one does not…

    a = LOAD ‘default.nyse_stocks’ USING org.apache.hcatalog.pig.HCatLoader();
    b = filter a by stock_symbol == ‘IBM’;
    c = group b all;
    d = foreach c generate avg(b.stock_volume);
    dump d;

    Collapse
    #56851

    Jon Petitta
    Participant

    I recently downloaded the sandbox as well. I too get the same error while trying to perform the tutorial for Pig.

    That file does not exist in that directory. Could there be an issue with the sandbox image, or maybe with the VirtualBox import?

    Jon

    Collapse
    #56811

    iandr413
    Moderator

    Hi Michael,
    Do you have this jar in /usr/lib/hive/lib? What step in the tutorial are you seeing this? My assumption it is in the PIG portion of the excercise. Does the PIG script fail or complete with this warning/error?

    Ian

    Collapse
    #56810

    iandr413
    Moderator

    HI Michael,
    I am sorry for your frustration. Please tell me which sandbox you are using, is it the 2.1 sandbox? Let me spend some time today researching what issue you are hitting and I will post my findings. Thank you.

    Ian

    Collapse
Viewing 13 replies - 1 through 13 (of 13 total)