Home Forums Oozie Issues with Oozie and HDP 1.2.3.1

Tagged: 

This topic contains 6 replies, has 4 voices, and was last updated by  Larry Liu 1 year, 5 months ago.

  • Creator
    Topic
  • #25459

    Snoot
    Participant

    Hi,

    We’re trying to set up HDP with Oozie according to the docs at http://docs.hortonworks.com/HDPDocuments/HDP1/HDP-1.2.3.1/bk_installing_manually_book/content/rpm-chap8-1.html

    The following command fails, saying that it doesn’t support version 2.x

    /usr/lib/oozie/bin/oozie-setup.sh -hadoop 2.x /usr/lib/hadoop -extjs /usr/share/HDP-oozie/ext-2.2.zip

    We worked around this issue and managed to install with the following command (changed to version to 0.20.200)

    /usr/lib/oozie/bin/oozie-setup.sh -hadoop 0.20.200 /usr/lib/hadoop -extjs /usr/share/HDP-oozie/ext-2.2.zip

    then created folder /etc/oozie/conf/action-conf, changed the config as described in the docs and added a few more changes (mostly so oozie will work with postgresql as its db) and we managed to bring it up.

    Now when we’re trying to start a job and we get HadoopAccessorException. In the logs we see something weird – it’s trying to access namenodename/ipconfig:8021 (I changed the name and ip):

    Caused by: java.net.ConnectException: Call to ec2-xx-xxx-xx-xx.compute-1.amazonaws.com/12.34.56.89:8021 failed on connection exception: java.net.ConnectException: Connection refused

    Any idea?

Viewing 6 replies - 1 through 6 (of 6 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #26066

    Larry Liu
    Moderator

    Thanks, Saggi. The example in the oozie package is still using incorrect port. Please correct it before running any example in the package.

    Larry

    Collapse
    #25964

    Snoot
    Participant

    Indeed – everything’s working. thanks!

    Collapse
    #25690

    tedr
    Moderator

    Hi Saggi,

    It’s good to hear that you’ve found the problem, does that mean that you’ve corrected for it as well, and are now up an running?

    Thanks,
    Ted.

    Collapse
    #25683

    Snoot
    Participant

    found the problem – the documentation and the accompanying config files tell the jobtracker to use port 50300 as the IPC port which is not the default 8021 we were telling oozie to use.

    Collapse
    #25681

    Snoot
    Participant

    Hi Tedr,

    We didn’t have any problem executing map-reduce jobs directly with the jobtracker and the external IP.

    I think that the problem begins with HDP’s bundled Oozie not being compatible with Hadoop 1.1.2.23 (in the same distribution)

    Collapse
    #25498

    tedr
    Member

    Hi Saggi,

    correct me if I’m wrong, but from the stack trace snippet it looks like you’re using the EC2 external names as the hostnames in your cluster. If that is the case that may be the root of the problem as it should be the internal name/ip for the cluster. the external name are used only when connecting to the nodes from the outside. the internal names usually begin with ‘ip-‘.

    Thanks,
    Ted.

    Collapse
Viewing 6 replies - 1 through 6 (of 6 total)