Home Forums Pig Running Apache Pig tutorial problems


This topic contains 0 replies, has 1 voice, and was last updated by  Jakub Stransky 5 months, 2 weeks ago.

  • Creator
  • #50777

    Jakub Stransky

    I am new to Apache Pig and have some experiences with Hadoop 2. I am having some difficulties with running “standard” pig tutorial – pig script1-hadoop.pig

    However because of cluster set up (users) – I had to modified an example a bit. Standard tutorial expects all files on / of HDFS. Which I cannot use in my case so I created /pig dir for that purpose

    drwxrwxrwx – hdfs hdfs 0 2014-03-31 11:15 /pig
    with the uploaded content

    -rw-r–r– 3 jakub hdfs 10408717 2014-03-31 10:41 /pig/excite.log.bz2
    I also modified the pig script script1-hadoop.pig as well to respect those changes as follows (mainly just for load and store commands):

    raw = LOAD ‘/pig/excite.log.bz2′ USING PigStorage(‘\t’) AS (user, time, query);

    STORE ordered_uniq_frequency INTO ‘/pig/script1-hadoop-results’ USING PigStorage();
    I am running pi script:

    [jakub@hadooptools pigtmp]$ pig script1-hadoop.pig
    but with no luck and getting error:

    2014-03-31 10:15:11,896 [main] ERROR org.apache.pig.tools.grunt.Grunt – You don’t have permission to perform the operation. Error from the server: Permission denied: user=jakub, access=WRITE, inode=”/”:hdfs:hdfs:drwxr-xr-x
    at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:234)
    at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:214)
    at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:158)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5202)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5184)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkAncestorAccess(FSNamesystem.java:5158)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:3405)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInt(FSNamesystem.java:3375)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3349)
    at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:724)
    at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:502)
    at …
    I am not quite sure why PIG script is trying to write into / on HDFS. I know that PIG can store some immediate results on HDFS so I modified pig.temp.dir property (/etc/pig/conf/pig.properties) and created location on HDFS /pig/tmp

    drwxrwxrwx – jakub hdfs 0 2014-03-31 11:15 /pig/tmp
    Anyone idea what might be wrong? Pig in local mode is ok.


You must be logged in to reply to this topic.