Home Forums HDFS Removing/Delete Nodes from HDP

This topic contains 5 replies, has 3 voices, and was last updated by  Sasha J 1 year, 5 months ago.

  • Creator
    Topic
  • #11680

    Hisham
    Member

    Hello,

    How can we remove a node (after adding it using HMC Add Nodes)?

    The case is that we have been testing expanding the cluster size by adding new nodes (cluster is hosted on Amazon EC2 instances). Now that we have finished testing, we have terminated these instances and now HMC and Nagios are still reporting them being down.

    I’ve tried manually removing them from Nagios configuration, everything went fine till we changed some HDFS settings on HMC, which caused some services to restart, and now all the dead nodes are back showing up in HMC and Nagios. So, seems like HMC is maintaining the nodes configuration somewhere else, any ideas?

Viewing 5 replies - 1 through 5 (of 5 total)

The topic ‘Removing/Delete Nodes from HDP’ is closed to new replies.

  • Author
    Replies
  • #11727

    Sasha J
    Moderator

    Sure.
    New release will be widely announced as soon as it will be ready for shipment.

    Thank you!
    Sasha

    Collapse
    #11726

    Hisham
    Member

    Thank you so much Sasha for the confirmation. I totally understand its on my own risk. For the same of completeness of this thread, can you please update it once node decommissioning is supported by HMC, so that anyone running into similar problem can benefit from it.

    Best Regards,
    Hisham

    Collapse
    #11725

    Sasha J
    Moderator

    Hisham
    yes, your procedure is correct.
    However, do not forget, this is unsupported changes in the internal database.
    Node decommissioning should be supported by HMC in the future implementation, but in the current release it is not supported.
    So, you can make the changes, but it is on your own risk.

    Thank you!

    Sasha

    Collapse
    #11718

    Hisham
    Member

    Thank you Ted. I’ve looked deeper into HMC and figured out how to change it in HMC’s db. Can you please confirm the following procedures:

    1- Remove the hosts from hmc sqlite database located at /var/db/hmc/data/data.db, basically delete from the tables: HostRoles and Hosts using the target host_name

    2- Force HMC to reconfigure nagios by changing it settings, personally I change the administrator password

    Anything else should be done?

    Collapse
    #11683

    tedr
    Member

    Hello,

    You seem to have done the correct procedure for removing a node. However the current version of HMC doesn’t support the decommissioning/removal of nodes. It is slated for a fix in a future version. The current work around is to do as you have done for removing the configurations from Nagios and such. Then for future starts and stops of services use the hadoop scripts rather than hmc.

    Ted.

    Collapse
Viewing 5 replies - 1 through 5 (of 5 total)