Home Forums HDP on Linux – Installation Centos 6.3 6 node physical install fail

This topic contains 21 replies, has 5 voices, and was last updated by  Seth Lyubich 1 year, 4 months ago.

  • Creator
    Topic
  • #17886

    Jason Morse
    Participant

    Hi I am trying to install a 6 node physical cluster using Ambari. We are getting the below error message in the log. We have a working 10 node cluster on VM’s and followed the ambari install guide. The install fails right at the registering hosts phase. This is on centos6.3.

    STDOUT
    Error in PREIN scriptlet in rpm package ambari-agent-1.2.1.2-1.x86_64
    sed: can’t read /etc/ambari-agent/conf/ambari-agent.ini: No such file or directory
    /bin/sh: /usr/sbin/ambari-agent: No such file or directory
    python: can’t open file ‘/usr/lib/python2.6/site-packages/ambari_agent/machine.py': [Errno 2] No such file or directory

Viewing 21 replies - 1 through 21 (of 21 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #32500

    Seth Lyubich
    Keymaster

    Hi Abhishek,

    Looks like you might have some conflicting packages installed. You can try to remove them and re-retry your installation. Also, this issue seems to be not related to original post. If you still not able to resolve the issue, can you please open new thread?

    Hope this helps,

    Thanks,
    Seth

    Collapse
    #31752

    I am getting the follow error while installing ambari on centOS6.3.

    err: /Stage[1]/Hdp::Snmp/Hdp::Package[snmp]/Hdp::Package::Process_pkg[snmp]/Package[net-snmp]/ensure: change from absent to present failed: Execution of ‘/usr/bin/yum -d 0 -e 0 -y install net-snmp’ returned 1: Error: Package: rpm-libs-4.8.0-12.el6.i686 (core-0)
    Requires: rpm = 4.8.0-12.el6
    Installed: rpm-4.8.0-27.el6.x86_64 (@anaconda-CentOS-201207061011.x86_64/6.3)
    rpm = 4.8.0-27.el6
    Available: rpm-4.8.0-12.el6.x86_64 (core-0)
    rpm = 4.8.0-12.el6
    Error: Package: glibc-2.12-1.7.el6.i686 (core-0)
    Requires: glibc-common = 2.12-1.7.el6
    Installed: glibc-common-2.12-1.80.el6_3.4.x86_64 (@centos-6.3-updates-x86_64/6.3)
    glibc-common = 2.12-1.80.el6_3.4
    Available: glibc-common-2.12-1.7.el6.x86_64 (core-0)
    glibc-common = 2.12-1.7.el6
    Error: Package: 4:perl-libs-5.10.1-115.el6.i686 (core-0)
    Requires: perl = 4:5.10.1-115.el6
    Installed: 4:perl-5.10.1-127.el6.x86_64 (@anaconda-CentOS-201207061011.x86_64/6.3)
    perl = 4:5.10.1-127.el6
    Available: 4:perl-5.10.1-115.el6.x86_64 (core-0)
    perl = 4:5.10.1-115.el6
    You could try using –skip-broken to work around the problem
    You could try running: rpm -Va –nofiles –nodigest

    Collapse
    #18436

    Robert
    Participant

    Hi Jason,
    I am glad it all worked out for you. Happy hadooping!

    Regards,
    Robert

    Collapse
    #18325

    Jason Morse
    Participant

    Looks like changing the default home on all servers is all it took. We now have a working cluster! Thanks for the help Robert!

    Collapse
    #18312

    Jason Morse
    Participant

    Ok I just tried the registration again and it worked! I’ll let you know how the rest of the install goes.

    Collapse
    #18285

    Jason Morse
    Participant

    No it did not get passed registration but I suspect it was right at the end. The part of the log below that I posted kind of looks like there may be an issue finding the hostname. Do you think it would help anything to wipe out the ambari server db by running ambari-server reset and trying the install over from scratch?

    Collapse
    #18256

    Robert
    Participant

    Hi Jason,
    Good, you got passed the initial issue with ambari-agent installation, now when you say it got further along, so did you get passed the registration and you clicked next to get into the install phase?

    Regards,
    Robert

    Collapse
    #18242

    Jason Morse
    Participant

    Thanks Robert. We changed the default home on 2 of the servers and were able to install ambari-agent using yum. After this I tried the Ambari install on just one of the hosts and it got further along but failed again later in the process.

    The ambari-server log shows below. It repeats that same last entry about 30 or so times then fails.

    09:14:21,193 INFO BSHostStatusCollector:55 – Request directory /var/run/ambari-server/bootstrap/1
    09:14:21,194 INFO BSHostStatusCollector:62 – HostList for polling on [hostname]
    09:14:22,211 INFO QueryImpl:152 – Executing resource query: {Host=null}
    09:14:22,211 INFO ClusterControllerImpl:92 – Using resource provider org.apache.ambari.server.controller.internal.HostResourceProvider for request type Host
    09:14:25,269 INFO QueryImpl:152 – Executing resource query: {Host=null}
    09:14:25,269 INFO ClusterControllerImpl:92 – Using resource provider org.apache.ambari.server.controller.internal.HostResourceProvider for request type Host

    Collapse
    #18206

    Robert
    Participant

    Hi Jason,
    I just confirmed, the /home is a hardcoded path, hence ambari would need to always write to this path. You should be able to file an enhancement request here:

    https://issues.apache.org/jira/browse/AMBARI

    so you can let the team know your use case. So the only workaround would probably be not to share the /home folder within your hadoop cluster machines.

    Regards,
    Robert

    Collapse
    #18204

    Robert
    Participant

    Hi Jason,
    Interesting setup. I will need to look some things over, but as of now I believe there is not a way to change that path where ambari needs to store some files in /home. I will double check. But, as far as a workaround, I assume you don’t want to write to that home folder since it’s basically a central home folder for all your machines. Disconnecting to that home folder and allowing each of your hadoop nodes to have a local home is not something you can possibly do?

    Regards,
    Robert

    Collapse
    #18131

    Jason Morse
    Participant

    It looks like the ambari-agent rpm needs to create some user home directories (puppet maybe others?) under /home. Our environment is set up with a shared home mounted at /home so we have the same environment on all servers. Is there any kind of work around for this type of set up that you guys have? Again thank you very much for your help with this.

    Dependencies Resolved

    ==========================================================================================
    Package Arch Version Repository Size
    ==========================================================================================
    Installing:
    ambari-agent x86_64 1.2.1.2-1 Updates-ambari-1.x 5.4 M

    Transaction Summary
    ==========================================================================================
    Install 1 Package(s)

    Total download size: 5.4 M
    Installed size: 21 M
    Is this ok [y/N]: y
    Downloading Packages:
    ambari-agent-1.2.1.2-1.x86_64.rpm | 5.4 MB 00:01
    Running rpm_check_debug
    Running Transaction Test
    Transaction Test Succeeded
    Running Transaction
    Error in PREIN scriptlet in rpm package ambari-agent-1.2.1.2-1.x86_64
    useradd: cannot create directory /home/puppet
    error: %pre(ambari-agent-1.2.1.2-1.x86_64) scriptlet failed, exit status 12
    error: install: %pre scriptlet failed (2), skipping ambari-agent-1.2.1.2-1
    Verifying : ambari-agent-1.2.1.2-1.x86_64 1/1

    Failed:
    ambari-agent.x86_64 0:1.2.1.2-1

    Collapse
    #18062

    Jason Morse
    Participant

    we do not have a gui on the servers so there is no browser on them. Also packagekit is disabled. I think I may know what the problem is. In the error message below when trying to install ambari-agent with yum there is an error that says:
    useradd: cannot create directory /home/puppet
    we have a shared home directory for all of our unix/linux servers that is mounted at /home because of this no directories can be created at /home. I think we may need to make /home available to the installer. I will try and update later.

    Collapse
    #17971

    Robert
    Participant

    Hi Jason,
    Interesting you can’t install manually. Do you have packagekit installed on the machine you are trying to install ambari-agent? If so, try disabling packagekit. As far as getting registered, hosts you need to log into ambari first with your browser prior to executing this rest call http://localhost:8080/api/v1/host on the same browser you just logged in with.
    Regards,
    Robert

    Collapse
    #17967

    Jason Morse
    Participant

    BTW thank you for your assistance. :)

    Collapse
    #17966

    Jason Morse
    Participant

    Below is the output when I try to install the agent using yum.

    [root@hostname ~]# yum install ambari-agent
    Loaded plugins: fastestmirror, security
    Loading mirror speeds from cached hostfile
    * base: centos.vipernetworksystems.com
    * epel: mirrors.servercentral.net
    * extras: mirror.team-cymru.org
    * updates: mirror.fdcservers.net
    Setting up Install Process
    Resolving Dependencies
    –> Running transaction check
    —> Package ambari-agent.x86_64 0:1.2.1.2-1 will be installed
    –> Finished Dependency Resolution

    Dependencies Resolved

    ================================================================================
    Package Arch Version Repository Size
    ================================================================================
    Installing:
    ambari-agent x86_64 1.2.1.2-1 Updates-ambari-1.x 5.4 M

    Transaction Summary
    ================================================================================
    Install 1 Package(s)

    Total download size: 5.4 M
    Installed size: 21 M
    Is this ok [y/N]: y
    Downloading Packages:
    ambari-agent-1.2.1.2-1.x86_64.rpm | 5.4 MB 00:10
    Running rpm_check_debug
    Running Transaction Test
    Transaction Test Succeeded
    Running Transaction
    Error in PREIN scriptlet in rpm package ambari-agent-1.2.1.2-1.x86_64
    useradd: cannot create directory /home/puppet
    error: %pre(ambari-agent-1.2.1.2-1.x86_64) scriptlet failed, exit status 12
    error: install: %pre scriptlet failed (2), skipping ambari-agent-1.2.1.2-1
    Verifying : ambari-agent-1.2.1.2-1.x86_64 1/1

    Failed:
    ambari-agent.x86_64 0:1.2.1.2-1

    Complete!

    Output of the python command (it is giving the correct hostname):

    [root@hostname home]# python
    Python 2.6.6 (r266:84292, Feb 22 2013, 00:00:18)
    [GCC 4.4.7 20120313 (Red Hat 4.4.7-3)] on linux2
    Type “help”, “copyright”, “credits” or “license” for more information.
    >>> import socket;
    >>> socket.getfqdn();
    ‘hostname.tsh.thomson.com’

    When I try to access the ambari database output in the browser I get the below message in the browser. I am not sure how to authenticate besides being logged into ambari when I go to that path.

    HTTP ERROR: 403

    Problem accessing /api/v1/hostname. Reason:

    Full authentication is required to access this resource

    Collapse
    #17963

    Robert
    Participant

    Hi Jason,
    You mention you installed the ambari-agent manually, I assume the failure you are referring to is with registering hosts and not the actual manual installation of ambari-agent? Out of the 6 nodes, I wonder if any of them registered. Can you check what currently is in ambari database? The following api call should list hosts that are registered: http://localhost:8080/api/v1/host
    replace localhost with your ambari host.

    NOTE: you must log in first prior to making the above call, or you will get an access denied message on your browser.

    Let us know what the output is.

    Also to help isolate, you may want to start placing one host first to verify if it registers. For example, just try to register the host that is hosting ambari-server. Secondly, what is the output of the following python call:

    import socket;
    socket.getfqdn();

    NOTE: from command line simply execute “python” in the command line to enter python shell.

    Regards,
    Robert

    Collapse
    #17958

    Jason Morse
    Participant

    Yes passwordless ssh is also enabled. I have also made sure that all of the prerequisite packages are installed. (scp, pdsh, rpm, etc.)

    Collapse
    #17957

    tedr
    Member

    Hi Jason,

    Is passwordless ssh for root set up from the machine your running ambari on to all other nodes in the cluster?

    Thanks,
    Ted.

    Collapse
    #17898

    Jason Morse
    Participant

    i removed the hostnames from the log.

    Collapse
    #17897

    Jason Morse
    Participant

    Below is from the logs. It almost looks like a problem with the ambari-agent rpm. I have tried installing the ambari-agent manually as well and it fails. The hosts file for all hosts has the fqdn of all hosts. We have also disabled selinux & iptables. Thanks for your help!

    This is out of the yum.log

    Mar 18 13:52:42 ambari-agent-1.2.1.2-1.x86_64: 100

    This is out of the ambari-server.log

    INFO:root:Copying files finished
    INFO:root:Running setup agent…
    INFO:root:Running ssh command ssh -o ConnectTimeOut=60 -o StrictHostKeyChecking=no -o BatchMode=yes -tt -i /var/run/ambari-server/bootstrap/1/sshKey root@ python /tmp/setupAgent.py DEV
    INFO:root:Running ssh command ssh -o ConnectTimeOut=60 -o StrictHostKeyChecking=no -o BatchMode=yes -tt -i /var/run/ambari-server/bootstrap/1/sshKey root@ python /tmp/setupAgent.py DEV
    INFO:root:Running ssh command ssh -o ConnectTimeOut=60 -o StrictHostKeyChecking=no -o BatchMode=yes -tt -i /var/run/ambari-server/bootstrap/1/sshKey root@ python /tmp/setupAgent.py DEV
    INFO:root:Running ssh command ssh -o ConnectTimeOut=60 -o StrictHostKeyChecking=no -o BatchMode=yes -tt -i /var/run/ambari-server/bootstrap/1/sshKey root@ python /tmp/setupAgent.py DEV
    INFO:root:Running ssh command ssh -o ConnectTimeOut=60 -o StrictHostKeyChecking=no -o BatchMode=yes -tt -i /var/run/ambari-server/bootstrap/1/sshKey root@ python /tmp/setupAgent.py DEV
    INFO:root:Running ssh command ssh -o ConnectTimeOut=60 -o StrictHostKeyChecking=no -o BatchMode=yes -tt -i /var/run/ambari-server/bootstrap/1/sshKey root@ python /tmp/setupAgent.py DEV
    INFO:root:Setup agent done for host , exitcode=2
    INFO:root:Setup agent done for host , exitcode=2
    INFO:root:Setup agent done for host , exitcode=2
    INFO:root:Setup agent done for host , exitcode=2
    INFO:root:Setup agent done for host , exitcode=2
    INFO:root:Setup agent done for host , exitcode=2
    INFO:root:Parallel ssh returns for setup agent. All failed hosts are: ['', '', '', '', '', '']. Failed on last step: ['', '', '', '', '', '']
    INFO:root:Running ssh command finished

    Collapse
    #17890

    Seth Lyubich
    Keymaster

    Hi Jason,

    Looks like Ambari agent did not install properly. Can you please check yum log and server log to see if you can get more information on what went wrong?

    Thanks,
    Seth

    Collapse
Viewing 21 replies - 1 through 21 (of 21 total)