Home Forums HDP on Linux – Installation Install Ambari from bash

Tagged: ,

This topic contains 10 replies, has 4 voices, and was last updated by  tedr 1 year, 8 months ago.

  • Creator
    Topic
  • #18563

    halacs
    Member

    Hi,

    I would like to install Ambari by using only bash on two Amazon virtual machine with CentOS6 but I have a big problem.

    I have found a little rest API and I can use it from bash by curl, but the install process is failed. Unfortunately, the error message could not help me As far as I know this API is not been documented well on the Internet. Am I right?

    You can see my bash script on the following url with which I do the installation: http://bvp6.iit.bme.hu/halacs/hadoop/install.txt
    You can see the status with the error message of the first failed task on the following url: http://bvp6.iit.bme.hu/halacs/hadoop/task2.txt

    Could you help me, which I do wrong? Maybe, are there any detailed documentation in connection with the Ambari API and how must do the install process?

    Thank you for the information.

Viewing 10 replies - 1 through 10 (of 10 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #20533

    tedr
    Member

    Hi Halacs,

    Thank you for letting us know that you solved the problem. It sounds like the timeouts you were getting was due to some slow responses on the network.

    Thanks again,
    TEd.

    Collapse
    #20531

    halacs
    Member

    Hi,

    I have found the solution: the example Ambari API calls at github must be wrong. I read the related part of source code and then I written a fully new API calls used curl which have already been working fine.

    There was another problem which occurred during the install phase (sometimes I got ‘TIMEDOUT’ messages), but I think it can be a temporary yum repository problem. I tried to get any kind of log by tcpflow but this issue was unexpectedly solved so it still be only a conjecture.

    Thank you for the advices.

    Halacs

    Collapse
    #19042

    Larry Liu
    Moderator

    Hi, halacs

    Your approach to install Ambari cluster is very different. But I still want to repeat Ted’s suggestion. Maybe you should set up passwordless and try again. From the error message, it seems the issue is with the configuration of ganglia.

    Larry

    Collapse
    #18902

    halacs
    Member

    Hi,

    Sorry, it seems to me that I wasn’t enough accurate, because I didn’t mention content of /etc/hosts file. Naturally, it is contains something like this (two hosts):
    1.2.3.4 ambariagent0
    5.6.7.8 ambariserver0

    “hostname” is a linux command, which set the system’s hostname, not the file content.

    Sorry for the ambiguity.

    Collapse
    #18884

    Seth Lyubich
    Keymaster

    Hi Halacs,

    I noticed that in your /etc/hosts file you only setting one hosts.
    vim /etc/hosts

    hostname ambariserver0

    Usually you should have all nodes in your cluster listed in /etc/hosts file so that all machines could be discovered correctly in the cluster.

    Can you please check if your name resolution works correctly? Also, do you have more logs?

    We also currently have three different install methods including manual installation. http://docs.hortonworks.com/HDPDocuments/HDP1/HDP-1.2.2/index.html . Can you please let us know why these methods do not work for you?

    Hope this helps,

    Thanks,
    Seth

    Collapse
    #18857

    halacs
    Member

    Hi,

    I use two Amazon EC2 virtual machine which are called ambariserver0 and ambariagent0. (this is only a test case).
    Here is my bash script:

    At the ambari server (install ambari-server and ambari-agent too)
    ———————————————————————————————

    setenforce 0
    sed -i “s/^SELINUX=.*/SELINUX=disabled/g” /etc/selinux/config

    # config DNS
    vim /etc/hosts

    hostname ambariserver0

    chkconfig iptables off
    /etc/init.d/iptables stop

    #rpm -Uvh http://public-repo-1.hortonworks.com/ambari/centos6/1.x/GA/ambari-1.2.0.1-1.el6.noarch.rpm
    wget http://public-repo-1.hortonworks.com/ambari/centos6/1.x/GA/ambari.repo && cp ambari.repo /etc/yum.repos.d

    yum install ambari-server

    # http://incubator.apache.org/ambari/1.2.1/installing-hadoop-using-ambari/content/ambari-chap6-1.html
    yum install ambari-agent

    ambari-server setup –silent

    sed -i “s/hostname=localhost/hostname=ambariserver0/g” /etc/ambari-agent/conf/ambari-agent.ini

    At the ambari agent (install ONLY ambari-agent)
    ——————————————————————

    setenforce 0
    sed -i “s/^SELINUX=.*/SELINUX=disabled/g” /etc/selinux/config

    # config DNS
    vim /etc/hosts

    hostname ambariagent0

    chkconfig iptables off
    /etc/init.d/iptables stop

    #rpm -Uvh http://public-repo-1.hortonworks.com/ambari/centos6/1.x/GA/ambari-1.2.0.1-1.el6.noarch.rpm
    wget http://public-repo-1.hortonworks.com/ambari/centos6/1.x/GA/ambari.repo && cp ambari.repo /etc/yum.repos.d

    # http://incubator.apache.org/ambari/1.2.1/installing-hadoop-using-ambari/content/ambari-chap6-1.html
    yum install ambari-agent

    sed -i “s/hostname=localhost/hostname=ambariserver0/g” /etc/ambari-agent/conf/ambari-agent.ini

    Collapse
    #18843

    Larry Liu
    Moderator

    Hi, Halacs

    I think it is a very interesting try. There is no instruction how to install HDP via bash by curl.

    I have checked your script. In the script, I didn’t find how ambari-server and ambari-agent got installed. Can you please provide the detail steps you have done?

    Larry

    Collapse
    #18832

    halacs
    Member

    So as I mentioned before, I have uploaded the log files from /var/ambari-* folders to the following server: http://bvp6.iit.bme.hu/halacs/hadoop/cluster/

    Because I need only a minimal hadoop install, I removed the ganglia and nagios install commands from my script. This modified script can be found the above url.

    Now, some install tasks are timed out and namenode has filed. Have you got any idea why?

    [root@ambariserver0 ~]# curl –user admin:admin -i -X PUT -d ‘{“ServiceInfo”: {“state” : “STARTED”}}’ http://$SERVER_HOST:8080/api/v1/clusters/c1/services
    HTTP/1.1 500 Server Error
    Set-Cookie: AMBARISESSIONID=1q1o2y89te9fk16uq5w9dnc2w0;Path=/
    Expires: Thu, 01 Jan 1970 00:00:00 GMT
    Content-Type: text/plain
    Content-Length: 326
    Server: Jetty(7.6.7.v20120910)

    {
    “status” : 500,
    “message” : “org.apache.ambari.server.controller.spi.SystemException: An internal system exception occurred: Invalid transition for servicecomponenthost, clusterName=c1, clusterId=1, serviceName=HDFS, componentName=NAMENODE, hostname=ambariserver0, currentState=INSTALL_FAILED, newDesiredState=STARTED”

    Collapse
    #18661

    halacs
    Member

    Hi Ted,

    I would like to use preinstalled agent instead of passwordless ssh so I didn’t set up passwordless ssh. Can it be trouble?
    I turned off iptables and set up the hosts file.
    By the way, if I use the web installer UI, the installation process will be finished well.
    Unfortunately, I cannot be able to see the log files, but I will check it on Monday.

    Collapse
    #18567

    tedr
    Member

    Hi Halacs,

    There is currently no detailed documentation on the Ambari API. From the error message you posted it looks like the ganglia install failed. As to why it failed, the error message doesn’t give much info, you can try looking at the ambari agent logs to, hopefully, get a better picture. Just to be sure, you have set up passwordless ssh between the ambari server host and all other hosts in the cluster correct? Also have you turned off iptables on all of the hosts, as well as making sure that the s3 security is allowing communication on all ports for ambari and hadoop?

    Thanks,
    Ted.

    Collapse
Viewing 10 replies - 1 through 10 (of 10 total)