Home Forums HDP on Linux – Installation Tips for guys who want try HDP

This topic contains 51 replies, has 16 voices, and was last updated by  Robert 1 year, 3 months ago.

  • Creator
    Topic
  • #5978

    Edy Liu
    Member

    1. think CentOS 6.x is better than CentOS 5.x because the php is 5.3
    tricky tips if you really want use CentOS 6.x.

    sed -i.bak ‘s/6.2/5.8/g’ /etc/redhat-release

    2. install net-snmp to avoid the snmpd.conf failed ?
    yum install -y net-snmp

    3. update the puppet. removed the php-pecl-json, php-pecl-json already included by default in php5.3. so you can safely remove the requirment.
    # line 11. remove the nagios-php-pecl-json
    /etc/puppet/master/modules/hdp-nagios/manifests/server/packages.pp

    4. seems the jdk met permission issue ?
    [root@hmhdp01 ~]# ls -l /var/www/html/downloads/
    total 166876
    -rwxr—– 1 root root 85292206 Jun 19 08:56 jdk-6u31-linux-i586.bin

    [root@hmhdp01 ~]# curl -I localhost/downloads/jdk-6u31-linux-x64.bin
    HTTP/1.1 403 Forbidden

    [root@hmhdp01 ~]# chown puppet /var/www/html/downloads/*
    [root@hmhdp01 ~]# curl -I localhost/downloads/jdk-6u31-linux-x64.bin
    HTTP/1.1 200 OK
    -rwxr—– 1 root root 85581913 Jun 19 08:56 jdk-6u31-linux-x64.bin

    5. still fight with HDP. not quite sure why the nagios/ganglia is a must for the installation.
    all looks fine now. but failed at last step , start nagios. maybe the configuration met issue. still debuging.

Viewing 21 replies - 31 through 51 (of 51 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #6481

    Leonid Fedotov
    Moderator

    Also, what kind of AWS instance you trying to use?
    HDP known to not work with “small”, as it have too low memory for running all subsystems.
    There is also known issue with timeouts during the installation, which leads to the exact error message you mentioned. Try to limit the amount of subsystems installed (like start from HDFS and MapReduce only)…

    Collapse
    #6465

    Sasha J
    Moderator

    Hi Sarath,

    can you confirm that the node that is running hmc is resolvable from all the hosts in the cluster?

    you can verify this by issuing:

    hostname -f

    then verify that name is resolvable from all nodes in the cluster using that name

    if its not you may have to make an entry in the /etc/hosts

    Let us know if you continue to have issues

    Sasha

    Collapse
    #6462

    I’m through with the “Add Nodes” step, here’s what I had to go through -
    1. Logged into the cluster node as root and opened HMC console (earlier was using console from a machine which is not in cluster) in Firefox 3.6.24
    2. On click of “Add Nodes” Firefox throws up a popup asking what to do with “addNodes.php”. Unable to resolve this issue, went ahead to install Chrome.
    3. After tweaking chrome (as it doesn’t run for root user), restarted the cluster creation process from HMC console and it’s through with “Add Nodes” step.

    Now I’m getting errors while selected services are getting installed. Same errors as mentioned by Binish above. The puppet report shows that puppet kick failed. Never worked on puppet so no idea what these errors mean.

    Please help getting through. hmc.log is big let me know how I can upload it.

    Collapse
    #6460

    Sasha,
    Created key using ssh-keygen on the node which I want to add. Copied the private key to a machine from where I’m running HMC console. Both these machines can SSH each other from root without password.

    But still it is not working. I also tried running HMC console directly on the node machine. Same issue persists.
    Let me know if you need any more logs/details.

    Collapse
    #6449

    Sasha J
    Moderator

    Binish,
    we need more details…
    What is the exact version, did you configure password-less connectivity, etc.
    uploading full hmc.log may also be helpful.

    Collapse
    #6448

    Sasha J
    Moderator

    Sarath,
    are you sure you have key working on the node you want to add?
    Your HMC node should eb able to connect to new one over ssh without a password. Please, check your ssh setup.

    Collapse
    #6446

    I am getting the following error in logs (hmc.log )in centos5 and this is an amazon machine
    So it fails the cluster installation first step
    [2012:06:27 15:32:27][INFO][PuppetInvoker][PuppetInvoker.php:314][waitForResults]: 0 out of 1 nodes have reported for txn 6-37-2
    [2012:06:27 15:32:32][INFO][PuppetInvoker][PuppetInvoker.php:314][waitForResults]: 0 out of 1 nodes have reported for txn 6-37-2

    any idea about this

    Collapse
    #6444

    Not able to complete the “Add Nodes” step.
    I’m trying with single cluster setup.Copied the private key of this machine and prepared a hostnames.txt file with single line containing FQDN of this machine.
    Logged in to HMC from another linux machine and supplied the above 2 files. On click of ‘Add Nodes’ it fails at ‘Finding Reachable Nodes’ with error ID 100.

    What is going wrong?

    hmc.txn.log:
    pdsh@algodb: module path “/usr/lib64/pdsh” insecure.
    pdsh@algodb: “/usr”: World writable and sticky bit is not set
    pdsh@algodb: Couldn’t load any pdsh modules

    hmc.log:
    [2012:06:27 14:23:47][ERROR][sequentialScriptExecutor][sequentialScriptRunner.php:251][]: Encountered total failure in transaction 100 while running cmd: /usr/bin/php ./addNodes/findSshableNodes.php with args: ALGOFUSION root 27 100 28 /var/run/hmc/clusters/ALGOFUSION/hosts.txt

    Collapse
    #6442

    It is working now. Issue was with SSH certificate. Recreated the certificate, reinstalled HMC and it’s up and running.

    Thanks for the quick help and support.

    Collapse
    #6441

    php-process installation is through, restarted hmc service, but the problem persists.

    rpm -qa|grep php:
    php-pdo-5.3.3-3.el6_1.3.x86_64
    php-common-5.3.3-3.el6_1.3.x86_64
    php-5.3.3-3.el6_1.3.x86_64
    php-process-5.3.3-3.el6_1.3.x86_64
    php-cli-5.3.3-3.el6_1.3.x86_64

    Collapse
    #6440

    Edy Liu
    Member

    hmm. strange, could you try yum install -y php-process

    then restart apache.

    if still doesn’t work. rpm -qa|grep php and post the output.

    Collapse
    #6439

    “curl -I http://localhost/hmc/html/index.php” returns nothing.

    This is what I see in httpd logs -
    127.0.0.1 – - [27/Jun/2012:16:21:59 +0530] “GET /hmc/html/index.php HTTP/1.1″ 500 – “-” “Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.24) Gecko/20111109 CentOS/3.6.24-3.el6.centos Firefox/3.6.24″

    Collapse
    #6438

    Edy Liu
    Member

    have you tried curl -I http://localhost/hmc/html/index.php check the return code ?

    check the httpd logs and you may get some clues.

    Collapse
    #6437

    I followed the steps given here and successfully installed HDP on CentOS 6.2. HMC service has also got started. But when I open http://localhost/hmc/html/index.php, it gives me a blank page.

    What am I missing here?

    -Sarath

    Collapse
    #6374

    Sasha J
    Moderator

    Hi guys,

    This looks like the server is resolving the puppet hostname to localhost

    make sure the fqdn is not associated with localhost.localdomain in the /etc/hosts

    if you are not sure, please post your the contents of your /etc/hosts here, and we can verify it for you

    -Sasha

    Collapse
    #6358

    Edy Liu
    Member

    seems so. i met similar issue before. I resolved similar issue by add PTR record to DNS.
    but a bit strange, i got (10.x.x.x) ip. you got loopback 127.0.0.1

    According to the guide. you’d better has dns server or put all the host ip-hostname to /etc/hosts

    my notes on CentOS 6.x,

    http://www.linuxdict.com/2012-06-auto-deploy-hadoop-cluster-with-hdp/

    Collapse
    #6357

    Wile Lee
    Member

    Hi Edy,

    thanks for your reply. The following message is found in the log but I don’t know it may be related to my problem?

    [2012:06:25 20:55:53][INFO][PuppetInvoker][PuppetInvoker.php:79][sendKick]: centos58-hdp: Kick failed with warning: peer certificate won’t be verified in this SSL session
    Host centos58-hdp failed: Error 403 on SERVER: Forbidden request: localhost.localdomain(127.0.0.1) access to /run/centos58-hdp [save] at line 1

    Collapse
    #6336

    Edy Liu
    Member

    I can’t tell from the log, you’d better review the /var/log/hmc/hmc.log check more details.

    my advice: tail -f /var/log/hmc/hmc.log from the deployment node and re-run the installation. you will get much clear what’s going on. ;)

    Collapse
    #6314

    Wile Lee
    Member

    Hi,

    the installation of hmc failed in the first step “Cluster Install” with the erorr, any help will be appreciated.

    Hortonworks Management Center

    Help

    Deploy Logs

    {
    “2″: {
    “nodeReport”: {
    “PUPPET_KICK_FAILED”: [],
    “PUPPET_OPERATION_FAILED”: [
    "centos58-hdp-1"
    ],
    “PUPPET_OPERATION_TIMEDOUT”: [
    "centos58-hdp-1"
    ],
    “PUPPET_OPERATION_SUCCEEDED”: []
    },
    “nodeLogs”: []
    },
    “56″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “57″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “58″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “61″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “63″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “64″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “66″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “68″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “70″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “71″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “73″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “74″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “75″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “79″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “80″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “81″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “85″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “89″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “90″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “94″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “95″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “96″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “100″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “101″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “102″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “103″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “114″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “115″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “116″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “117″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “119″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “120″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “121″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “123″: {
    “nodeReport”: [],
    “nodeLogs”: []
    },
    “124″: {
    “nodeReport”: [],
    “nodeLogs”: []
    }
    }

    Deployment Progress

    Cluster install
    Failed
    HDFS start
    Pending
    HDFS test
    Pending
    MapReduce start
    Pending
    MapReduce test
    Pending
    ZooKeeper start
    Pending
    ZooKeeper test
    Pending
    HBase start
    Pending
    HBase test
    Pending
    Pig test
    Pending
    Sqoop test
    Pending
    Oozie start
    Pending
    Oozie test
    Pending
    Hive/HCatalog start
    Pending
    Hive/HCatalog test
    Pending
    Templeton start
    Pending
    Templeton test
    Pending
    Dashboard start
    Pending
    Ganglia start
    Pending
    Nagios start
    Pending

    Failed to finish setting up the cluster.
    Take a look at the deploy logs to find out what might have gone wrong.Reinstall Cluster
    Hortonworks © 2012

    Collapse
    #5983

    Sasha J
    Moderator

    Hi Edy,

    Thanks for all the great info. We just wanted to remind everyone that the officially supported targets are RHEL/CentOS 5.x

    While you may be able to get the distro to work with 6.x, it is not currently a supported target.

    Thanks again for your interest in HDP!

    Sasha

    Collapse
    #5979

    Edy Liu
    Member

    if you failed for the ruby dependence.

    try
    yum install http://passenger.stealthymonkeys.com/rhel/6/passenger-release.noarch.rpm

    if you failed installation ? and can’t uninstall/reinstall ?

    yum remove hmc && yum install -y hmc && service hmc start

    then you can re-launch the installation.

    Collapse
Viewing 21 replies - 31 through 51 (of 51 total)