Home Forums HDP on Linux – Installation Ambari failing to register any hosts, no errors in agent or server logs

This topic contains 6 replies, has 3 voices, and was last updated by  Tim Benninghoff 1 year, 4 months ago.

  • Creator
    Topic
  • #29037

    Tim Benninghoff
    Participant

    This is my second attempt at setting up a Hadoop cluster via Ambari. All of the planned nodes are Centos 6.4 64bit from the minimal install. The NameNode is a new machine built from scratch, but all of the other nodes are being re-used from a previous failed setup of the cluster.

    To refresh the nodes that are being reused, I did the following based on tedr’s recommendation in this thread (http://hortonworks.com/community/forums/topic/registration-with-the-ambari-server-failed/):

    ambari-agent stop
    yum erase ambari-agent
    rm -rf/var/lib/ambari-agent
    yum install ambari-agent
    (configure ambari-agent.ini to point to ambari-server)
    ambari-agent start

    Some other information about the environment…I installed the agents manually and also made sure that password-less ssh works from what will be the namenode/ambari-server to all of the other nodes.

    So, what’s happening is that when I’m trying to setup the new cluster, none of the nodes are registering in the Confirm Hosts stage. Looking at all of the logs, ambari-server and ambari-agents, I see no errors whatsoever. I made sure I’m using FQDN, and taking a hint from an ambari mailing list thread, I tried the following:
    curl -u admin: http://NAMENODE.domain.com:8080/api/v1/hosts

    I see the following returned:

    {
    “href” : “http://NAMENODE.domain.com:8080/api/v1/hosts”,
    “items” : [
    {
    "href" : "http://NAMENODE.domain.com:8080/api/v1/hosts/SNAMENODE.domain.com",
    "Hosts" : {
    "host_name" : "SNAMENODE.domain.com"
    }
    },
    {
    "href" : "http://NAMENODE.domain.com:8080/api/v1/hosts/DATANODE1.domain.com",
    "Hosts" : {
    "host_name" : "DATANODE1.domain.com"
    }
    },
    {
    "href" : "http://NAMENODE.domain.com:8080/api/v1/hosts/DATANODE2.domain.com",
    "Hosts" : {
    "host_name" : "DATANODE2.domain.com"
    }
    },
    {
    "href" : "http://NAMENODE.domain.com:8080/api/v1/hosts/DATANODE3.domain.com",
    "Hosts" : {
    "host_name" : "DATANODE3.domain.com"
    }
    },
    {
    "href" : "http://NAMENODE.domain.com:8080/api/v1/hosts/NAMENODE.domain.com",
    "Hosts" : {
    "host_name" : "NAMENODE.domain.com"
    }
    }
    ]
    }

    This is what I would expect to see based on the FQDN I added for the Target Hosts in the Install Options step. Without any errors to work from, I’m stumped as to what to try next.
    Any suggestions for getting some error logs generated or something to hint as to why the host registration is failing?

Viewing 6 replies - 1 through 6 (of 6 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #30466

    Tim Benninghoff
    Participant

    I agree that it’s a good idea to double and triple check the basics when troubleshooting, but sadly, no, firewalls were not blocking anything.

    Collapse
    #30448


    Member

    I hate to state the obvious but did you check that a firewall is not blocking you

    Collapse
    #29074

    Sasha J
    Moderator

    Good to know!

    Thank you!
    Sasha

    Collapse
    #29061

    Tim Benninghoff
    Participant

    I found my problem. For some reason I had just got in the habit of accessing Ambari via the IP address (ie. xx.xx.xx.xx:8080) in my browser instead of using the FQDN for the Ambari server URL. When I switched over to using the FQDN URL for Ambari, all of the hosts registered immediately.

    Collapse
    #29060

    Tim Benninghoff
    Participant

    Hi, Sasha. Thanks for the response.

    Yes, I did install all agents manually and I did select ‘manual registration’ which pops up a couple of warnings about installing agents manually.
    When I say I configured SSH, I did so manually outside of the Ambari install. It had been my experience during my first installation attempt via Ambari that various components of the cluster, (Ganglia, Nagios, HBASE) ran into a great many problems with SSH blocking communication.

    Collapse
    #29038

    Sasha J
    Moderator

    Tim,
    please clarify:
    you installed all agents manually and then run installation with all the defaults settingsin UI?
    you have to check “manual registration” for this.
    And you do not need to have SSH configured in this case.

    Please, confirm you use “manual” process in the UI.

    Thank you!
    Sasha

    Collapse
Viewing 6 replies - 1 through 6 (of 6 total)