Home Forums Oozie JA018 Cannot run program

This topic contains 3 replies, has 2 voices, and was last updated by  Amudhan K 3 months ago.

  • Creator
    Topic
  • #52193

    Gwenael Le Barzic
    Participant

    Hello !

    I contact you because we currently have a problem on our cluster in HDP 2.0 with kerberos installed.
    We have Oozie server installed on one server and we have 6 data nodes.

    We tried to launch a workflow which needs to launch a shell script in bash, but we keep encountering the following error message :
    0000014-140423121102029-oozie-oozi-W@myAction http://<NAMENODE_FQDN>:<PORT>/proxy/application_1398179473939_0031/ JA018 Cannot run program "myShell.sh" (in directory "/var/opt/data/flat/data003/hadoop/yarn/local/usercache/my_user/appcache/application_1398179473939_0031/container_1398179473939_0031_01_000001"): error=13, Permission denied job_1398179473939_0031 FAILED/KILLED myAction 0 <NAMENODE_FQDN>:<PORT> shell 2014-04-23 12:53:17 GMT ERROR 2014-04-23 12:53:29 GMT

    Here is the command line to launch the workflow :
    oozie job -oozie=http://<HOSTNAME_OOZIE_SERVER>:<PORT>/oozie -debug -verbose -config /home/my_user/job.properties -nocleanup -run

    Here is the content of job.properties :
    #nameNode Adress
    nameNode=hdfs://<NAMENODE_FQDN>:<PORT>
    #jobTracker adress
    jobTracker=<NAMENODE_FQDN>:<PORT>
    #Yarn queue
    queueName=my_queue
    #adress where the workflow is deployed
    oozie.wf.application.path=hdfs://<NAMENODE_FQDN>:<PORT>/user/my_user/
    oozie.use.system.libpath=true
    #adresse du serveur oozie
    oozieHost=http://<HOSTNAME_OOZIE_SERVER>:<PORT>/oozie

    Here is the workflow.xml :
    <workflow-app xmlns="uri:oozie:workflow:0.5" name="myWorkflow">
    <start to="myAction" />
    <action name="myAction">
    <shell xmlns="uri:oozie:shell-action:0.2">
    <job-tracker>${jobTracker}</job-tracker>
    <name-node>${nameNode}</name-node>
    <configuration>
    <property>
    <name>mapred.job.queue.name</name>
    <value>${queueName}</value>
    </property>
    </configuration>
    <exec>myShell.sh</exec>
    <file>/user/myuser/oozie/workflow/shell/myShell.sh#myShell.sh</file>
    <capture-output />
    </shell>
    <ok to="end" />
    <error to="fail" />
    </action>
    <kill name="fail">
    <message>Workflow failed, error
    message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end" />
    </workflow-app>

    I will continue my description in my next post.

    Best regards.

    Gwenael Le Barzic

Viewing 3 replies - 1 through 3 (of 3 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #54248

    Amudhan K
    Participant

    Also check whether the user JA018 or the user from which you are running oozie have all the access to your HDFS.

    Collapse
    #54246

    Amudhan K
    Participant

    Hello Gwenael Le Barzic,
    The problem is not with your workflow or job.properties. The error shows that the permission is denied for the JA018. You need to add the proxy user properties to the hadoop core-site.xml. the properties which you need to add are

    <property>
    <name>hadoop.proxyuser.JA018.hosts</name>
    <value>*</value>
    </property>
    <property>
    <name>hadoop.proxyuser.JA018.groups</name>
    <value>*</value>
    </property>

    after adding these properties to your cluster and execute the refresh nodes command or restart your cluster.

    Collapse
    #52194

    Gwenael Le Barzic
    Participant

    Here is what I already tried :
    1. Check on each data nodes the authorization on each folder “/var/opt/data/flat/data<XXX>/hadoop/yarn/local/usercache/my_user/appcache” : everything is OK
    2. Check the content of my file myShell.sh to check if there is bad things inside : OK
    3. The workflow.xml file is located in HDFS in /user/my_user/ with the following authorization :
    -rw-rw-r-- 3 my_user my_group 957 2014-04-23 15:22 /user/my_user/workflow.xml

    4. The shell myshell.sh is located in HDFS in /user/myuser/oozie/workflow/shell/ with the following rights :
    -rw-rw-r-- 3 my_user my_group 3936 2014-04-22 11:34 /user/myuser/oozie/workflow/shell/myshell.sh

    Best regards.

    Gwenael Le Barzic

    Collapse
Viewing 3 replies - 1 through 3 (of 3 total)