The Hortonworks Community Connection is now live. A completely rebuilt Q&A forum, Knowledge Base, Code Hub and more, backed by the experts in the industry.

You will be redirected here in 10 seconds. If your are not redirected, click here to visit the new site.

The legacy Hortonworks Forum is now closed. You can view a read-only version of the former site by clicking here. The site will be taken offline on January 31,2016

HDP on Linux – Installation Forum

Ambari agent does not work in installation

  • #48789
    Begum Tuncer

    I tried to perform an automated installation on a cluster but even after the fifth trial it failed again with the same stange error. The hosts are registered successfully, but when the wizard installs the services it ends up with some warnings (yellow) in most of the hosts about hdfs&mapreduce checks and starting services. It shows the namenode host green, but in fact no services are installed in this host.
    Ambari server or agent logs tells me nothing remarkable:

    From ambari-server.log:
    11:55:05,546 INFO [Thread-30] ActionScheduler:434 – Not a service component, assuming its an action
    org.apache.ambari.server.ServiceComponentNotFoundException: ServiceComponent not found, clusterName=Example, serviceName=HDFS, serviceComponentName=HDFS_SERVICE_CHECK
    at org.apache.ambari.server.state.ServiceImpl.getServiceComponent(
    at org.apache.ambari.server.actionmanager.ActionScheduler.scheduleHostRole(
    at org.apache.ambari.server.actionmanager.ActionScheduler.doWork(
    12:05:18,611 WARN [Thread-30] ActionScheduler:187 – Operation completely failed, aborting request id:2

    From namenode’s ambari-agent.log:
    INFO 2014-02-17 11:35:56,638 – SSL connection established. Two-way SSL authentication is turned off on the server.
    INFO 2014-02-17 11:35:56,752 – Registered with the server with {u’exitstatus’: 0, u’response’: u’OK’, u’responseId’: 0, u’statusCommands’: []}
    INFO 2014-02-17 11:35:56,752 – Got status commands on registration []
    INFO 2014-02-17 11:35:56,752 – Response from server = OK
    INFO 2014-02-17 11:36:06,763 – Sending heartbeat with response id: 0 and timestamp: 1392629766762. Command(s) in progress: False. Components mapped: False
    INFO 2014-02-17 11:36:09,036 – Host check report at /var/lib/ambari-agent/data/hostcheck.result
    INFO 2014-02-17 11:36:09,036 – Removing old host check file at /var/lib/ambari-agent/data/hostcheck.result
    INFO 2014-02-17 11:36:09,037 – Creating host check file at /var/lib/ambari-agent/data/hostcheck.result
    INFO 2014-02-17 11:36:09,057 – No commands sent from the Server.

    So when the installation of services ends up with warnings, ambari ui stucks as “Loading” if i try to continue and complete installation. At that time, I get some logs like this in ambari-server.log:
    14:57:25,059 ERROR [pool-2-thread-8] JMXPropertyProvider:487 – Caught exception getting JMX metrics : Connection refused

    How can I complete the installation of all services successfully in all hosts? Why is ambari-agent of some hosts work correctly but others do nothing after registering the host?

    OS: Centos 6.5
    Ambari version:
    Stack: HDP 1.3.2

  • Author
  • #48843
    Kenny Zhang

    Hi Begum,

    Could you please share the output of the following commands from the namenode host?

    # cat /var/run/hadoop/hdfs/
    # /usr/jdk64/jdk1.6.0_31/bin/jps


    Begum Tuncer

    Hi Kenny,
    Thanks for reply.
    It seems that none of the services are installed on namenode host. Even jdk is not installed.
    # /usr/jdk64/jdk1.6.0_31/bin/jps
    -bash: /usr/jdk64/jdk1.6.0_31/bin/jps: No such file or directory
    # rpm -qa | grep hadoop
    Only ambari-agent is installed on that host, nothing else.

    Kenny Zhang

    Hi Begum,

    This is interesting and make sense.
    Since the jdk is not even installed, so the error (JMX connection refused) is expected.
    Could you please issue the following commands and let me know what’s the result?
    # ambari-server reset
    # ambari-server setup

    You can refer to the following document:

    Please let me know if you can successfully install JDK.


    Begum Tuncer

    I do ambari-server reset & setup but it did not help.
    I realized that on the problematic hosts, there is no HDP repo in /etc/yum.repos.d
    So I tried manually installing ambari server and agents, and manually copying repos to all hosts and then start installation. But again the same problem, no services install on the problematic hosts. In fact, “nothing” happened on them during the installation, i got the same logs as i sent before.
    Any ideas?

    Kenny Zhang

    Hi Begum,

    Yes, I think we need to fix the jdk problem first. The jdk is downloaded during you run “ambari-server setup” on the ambari server. You can verify this by looking at /var/lib/ambari-server/resources/jdk-6u31-linux-x64.bin.

    If you fails to download the jdk.bin, ambari will not be able to install it on all the hosts, which result in the HDP installation fail. Please also ensure you are running ambari-agent as root and has the privilege to install application to the system.

    Hope this helps.


    Begum Tuncer

    Hi again,
    I think I can not explain the situation good.
    jdk-6u31-linux-x64.bin is correctly downloaded to ambari-server machine when i do “ambari-server setup”. I can see it under /var/lib/ambari-server/resources/.
    I start HDP installation, it finished with some warnings.
    I lookeed at all the hosts (There are 11 slaves and 4 masters). All hosts installed necessary services except one master (namenode) and one slave.
    jdk is installed on all machines, except these two machines.
    root user runs ambari-agent services on all machines.
    There are no error logs in these two machines ambari-agent.logs.
    I’ll try to install jdk’s manually on these two machines and try the HDP installation again.

    Begum Tuncer

    After some more try, i realized that the problem is general. I made the installation in completely different environment (fully fresh installation of Centos 6.5 virtual machines) and the problem occured very similarly. This time, none of the services are installed. Here is the complete ambari-server.log:

    In the same environment I made a successful installation when I choose HDP 1.3.3 stack version.

    I think there is problem between HDP 1.3.2 version and ambari-server/agent 1.4.4 version.

The forum ‘HDP on Linux – Installation’ is closed to new topics and replies.

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.