Install Ambari from bash

Hi,

I would like to install Ambari by using only bash on two Amazon virtual machine with CentOS6 but I have a big problem.

I have found a little rest API and I can use it from bash by curl, but the install process is failed. Unfortunately, the error message could not help me As far as I know this API is not been documented well on the Internet. Am I right?

You can see my bash script on the following url with which I do the installation: http://bvp6.iit.bme.hu/halacs/hadoop/install.txt
You can see the status with the error message of the first failed task on the following url: http://bvp6.iit.bme.hu/halacs/hadoop/task2.txt

Could you help me, which I do wrong? Maybe, are there any detailed documentation in connection with the Ambari API and how must do the install process?

Thank you for the information.

Topic

[tedrMember]

Hi Halacs,

Thank you for letting us know that you solved the problem. It sounds like the timeouts you were getting was due to some slow responses on the network.

Thanks again,
TEd.

[halacsMember]

Hi,

I have found the solution: the example Ambari API calls at github must be wrong. I read the related part of source code and then I written a fully new API calls used curl which have already been working fine.

There was another problem which occurred during the install phase (sometimes I got ‘TIMEDOUT’ messages), but I think it can be a temporary yum repository problem. I tried to get any kind of log by tcpflow but this issue was unexpectedly solved so it still be only a conjecture.

Thank you for the advices.

Halacs

[Larry LiuModerator]

Hi, halacs

Your approach to install Ambari cluster is very different. But I still want to repeat Ted’s suggestion. Maybe you should set up passwordless and try again. From the error message, it seems the issue is with the configuration of ganglia.

Larry

[halacsMember]

Hi,

Sorry, it seems to me that I wasn’t enough accurate, because I didn’t mention content of /etc/hosts file. Naturally, it is contains something like this (two hosts):
1.2.3.4 ambariagent0
5.6.7.8 ambariserver0

“hostname” is a linux command, which set the system’s hostname, not the file content.

Sorry for the ambiguity.

[SefMember]

Hi Halacs,

I noticed that in your /etc/hosts file you only setting one hosts.
vim /etc/hosts

hostname ambariserver0

Usually you should have all nodes in your cluster listed in /etc/hosts file so that all machines could be discovered correctly in the cluster.

Can you please check if your name resolution works correctly? Also, do you have more logs?

We also currently have three different install methods including manual installation. http://docs.hortonworks.com/HDPDocuments/HDP1/HDP-1.2.2/index.html . Can you please let us know why these methods do not work for you?

Hope this helps,

Thanks,
Seth

[halacsMember]

Hi,

I use two Amazon EC2 virtual machine which are called ambariserver0 and ambariagent0. (this is only a test case).
Here is my bash script:

At the ambari server (install ambari-server and ambari-agent too)
———————————————————————————————

setenforce 0
sed -i “s/^SELINUX=.*/SELINUX=disabled/g” /etc/selinux/config

# config DNS
vim /etc/hosts

hostname ambariserver0

chkconfig iptables off
/etc/init.d/iptables stop

#rpm -Uvh http://public-repo-1.hortonworks.com/ambari/centos6/1.x/GA/ambari-1.2.0.1-1.el6.noarch.rpm
wget http://public-repo-1.hortonworks.com/ambari/centos6/1.x/GA/ambari.repo && cp ambari.repo /etc/yum.repos.d

yum install ambari-server

# http://incubator.apache.org/ambari/1.2.1/installing-hadoop-using-ambari/content/ambari-chap6-1.html
yum install ambari-agent

ambari-server setup –silent

sed -i “s/hostname=localhost/hostname=ambariserver0/g” /etc/ambari-agent/conf/ambari-agent.ini

At the ambari agent (install ONLY ambari-agent)
——————————————————————

setenforce 0
sed -i “s/^SELINUX=.*/SELINUX=disabled/g” /etc/selinux/config

# config DNS
vim /etc/hosts

hostname ambariagent0

chkconfig iptables off
/etc/init.d/iptables stop

#rpm -Uvh http://public-repo-1.hortonworks.com/ambari/centos6/1.x/GA/ambari-1.2.0.1-1.el6.noarch.rpm
wget http://public-repo-1.hortonworks.com/ambari/centos6/1.x/GA/ambari.repo && cp ambari.repo /etc/yum.repos.d

# http://incubator.apache.org/ambari/1.2.1/installing-hadoop-using-ambari/content/ambari-chap6-1.html
yum install ambari-agent

sed -i “s/hostname=localhost/hostname=ambariserver0/g” /etc/ambari-agent/conf/ambari-agent.ini

[Larry LiuModerator]

Hi, Halacs

I think it is a very interesting try. There is no instruction how to install HDP via bash by curl.

I have checked your script. In the script, I didn’t find how ambari-server and ambari-agent got installed. Can you please provide the detail steps you have done?

Larry

[halacsMember]

So as I mentioned before, I have uploaded the log files from /var/ambari-* folders to the following server: http://bvp6.iit.bme.hu/halacs/hadoop/cluster/

Because I need only a minimal hadoop install, I removed the ganglia and nagios install commands from my script. This modified script can be found the above url.

Now, some install tasks are timed out and namenode has filed. Have you got any idea why?

[root@ambariserver0 ~]# curl –user admin:admin -i -X PUT -d ‘{“ServiceInfo”: {“state” : “STARTED”}}’ http://$SERVER_HOST:8080/api/v1/clusters/c1/services
HTTP/1.1 500 Server Error
Set-Cookie: AMBARISESSIONID=1q1o2y89te9fk16uq5w9dnc2w0;Path=/
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Content-Type: text/plain
Content-Length: 326
Server: Jetty(7.6.7.v20120910)

{
“status” : 500,
“message” : “org.apache.ambari.server.controller.spi.SystemException: An internal system exception occurred: Invalid transition for servicecomponenthost, clusterName=c1, clusterId=1, serviceName=HDFS, componentName=NAMENODE, hostname=ambariserver0, currentState=INSTALL_FAILED, newDesiredState=STARTED”

[halacsMember]

Hi Ted,

I would like to use preinstalled agent instead of passwordless ssh so I didn’t set up passwordless ssh. Can it be trouble?
I turned off iptables and set up the hosts file.
By the way, if I use the web installer UI, the installation process will be finished well.
Unfortunately, I cannot be able to see the log files, but I will check it on Monday.

[tedrMember]

Hi Halacs,

There is currently no detailed documentation on the Ambari API. From the error message you posted it looks like the ganglia install failed. As to why it failed, the error message doesn’t give much info, you can try looking at the ambari agent logs to, hopefully, get a better picture. Just to be sure, you have set up passwordless ssh between the ambari server host and all other hosts in the cluster correct? Also have you turned off iptables on all of the hosts, as well as making sure that the s3 security is allowing communication on all ports for ambari and hadoop?

Thanks,
Ted.

[Author]

Replies