Home Forums HDP on Linux – Installation HDFS start failed

This topic contains 11 replies, has 4 voices, and was last updated by  tedr 1 year, 5 months ago.

  • Creator
    Topic
  • #11774

    Jinsong Yin
    Member

    I have got an error:HDFS start failed. And i have uploaded the script resulting file(Jinsong.check.sh.log) and the hmc.log(Jinsong.hmc.log), any advice is appreciated.

    thanks a lot!
    Jinsong Yin

Viewing 11 replies - 1 through 11 (of 11 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #12087

    tedr
    Member

    Lindsay,

    The only thing I can suggest at this point is to retry the install without doing the uninstall. I’ve had times during an install that one of the items failed, but on a retry it passed. Also since you are using HDP2.0 I’ve really helped you here more than I should have. This question should have been posted in the HDP2.0 alpha feedback section of this forum.

    Ted.

    Collapse
    #12084

    Lindsay Weir
    Member

    Thanks Ted. I did make progress but it now fails on the Oozie start phase. hw.tar has been uploaded with logs.

    Lindsay

    Collapse
    #12070

    tedr
    Member

    Hi Lindsay,

    From the namenode logs, it looks like your name node did not get formatted during installation. You can correct for this by:

    * run the installation till if fails
    * don’t do the uninstall that HMC recommends
    * instead close the browser
    * on the console/terminal in the box where you are running the installer issue the following commands as root:
        su hdfs
        hadoop namenode -format
        exit
        yum erase hmc puppet
        yum install hmc
        service hmc start
    * now do the installation as usual

    I hope that this gets you to a good install
    Ted

    Collapse
    #12066

    Lindsay Weir
    Member

    Thanks Ted. I have uploaded the logs – hdfs_logs.tar

    Lindsay

    Collapse
    #12041

    tedr
    Member

    Hi Lindsay,

    the namenode logs are usually in /var/log/hadoop/hdfs and have ‘namenode’ in the filename and end with ‘.log’
    something like: hadoop-<user>-namenode-<hostname>.log

    Ted.

    Collapse
    #12040

    Lindsay Weir
    Member

    Which are they? Thx

    Collapse
    #12039

    tedr
    Member

    Hi Lindsay,

    Could you also send along the namenode logs?

    Ted.

    Collapse
    #12034

    Lindsay Weir
    Member

    iptables are turned off on all nodes.

    [root@hmc11 ~]# service iptables status
    iptables: Firewall is not running.

    Thanks for SELinux, I corrected this on all the nodes and rebooted all the nodes and tried again.

    [root@hmc11 ~]# more /etc/sysconfig/selinux

    # This file controls the state of SELinux on the system.
    # SELINUX= can take one of these three values:
    # enforcing – SELinux security policy is enforced.
    # permissive – SELinux prints warnings instead of enforcing.
    # disabled – No SELinux policy is loaded.
    SELINUX=disabled
    # SELINUXTYPE= can take one of these two values:
    # targeted – Targeted processes are protected,
    # mls – Multi Level Security protection.
    SELINUXTYPE=targeted

    [root@hmc11 ~]# getenforce
    Disabled

    I uninstalled:

    yum -y erase puppet hmc
    yum -y install hmc
    service hmc start

    and then ran it again but it fails in the same place. I have uploaded the check.sh and logs again in the hw2.tar upload.

    Thanks again

    Lindsay

    Collapse
    #12030

    tedr
    Member

    Hi Lindsay,

    Since you are working with HDP2.0 you should post this questions there. But, in looking at the logs you posted, if looks like you need to disable SElinux on all of the computers on the cluster and turn of iptables on hmc11. Then retry the installation.

    Let me know if this helps.
    Ted.

    Collapse
    #12029

    Lindsay Weir
    Member

    I am also running into the same problem for version 2.0. I have uploaded a tar file with the check.sh, hmc.log and puppet log files (file called hw.tar). Passwordless SSH works, NTP running, puppet kick tests work from the hmc to all the nodes. Local hosts file resolves each node correctly. I have uninstalled and run the installation several times and it still fails on the same location.

    Thanks,

    Lindsay

    Collapse
    #11835

    Sasha J
    Moderator

    Jinsong,
    It looks like your problem coming from timeouts during the installation.

    [timedoutnodes] => Array
    (
    [0] => datanode2test.localdomain
    [1] => datanode1test.localdomain
    [2] => secondarynamenodetest.localdomain
    )

    Please, do the following commands:
    1. on all nodes: yum erase hmc puppet
    2. on HMC node: yum install hmc
    3. on HMC node: service hmc start

    then connect to HMC service trough UI and rerun your installation.
    Make sure you have all the RPMs installed on the nodes during the process.

    Thank you!
    Sasha

    Collapse
Viewing 11 replies - 1 through 11 (of 11 total)