Home Forums HDP on Linux – Installation Can I seperate Hadoop Master Nodes from DataNodes ?

This topic contains 3 replies, has 2 voices, and was last updated by  tedr 1 year, 9 months ago.

  • Creator
    Topic
  • #13210

    For some reason, I want to configure my hadoop cluster with master node seperated from datanodes. What I want are :
    Clusters:10 nodes
    Master Nodes :Host001,Host002,Host003 ===>3 master nodes
    Slave Nodes:Host004,Host005,Host006,Host007,Host008,Host009====>7 slave nodes
    Host001 :NameNode+Zookeeper1,
    Host002: SecondNomenode+Jobtracker+Zookeeper2 ,
    Host003:HBase+Zookeeper3,
    Host004:DataNode+HDFS+RegionServer
    Host005:DataNode+HDFS+RegionServer
    Host006:DataNode+HDFS+RegionServer
    Host007:DataNode+HDFS+RegionServer
    Host008:DataNode+HDFS+RegionServer
    Host009:DataNode+HDFS+RegionServer
    Host010:DataNode+HDFS+RegionServer

    But I found HMC can not configure Master nodes separated from Slave Nodes
    What I got from HMC deployment were :
    Host001 :NameNode+Zookeeper1, +DataNode+HDFS+Regionserver
    Host002: SecondNomenode+Jobtracker+Zookeeper2 ,+DataNode+HDFS+Regionserver
    Host003:HBase+Zookeeper3,+DataNode+HDFS+Regionserver
    Host004:DataNode+HDFS+Regionserver
    Host005:DataNode+HDFS+Regionserver
    ………..
    Host010:DataNode+HDFS+Regionserver

    Question:
    How can I get what I want by configuring HMC or something?
    Can I separate Hadoop Master Nodes from DataNodes completly?

    Regards,
    Jeff

Viewing 3 replies - 1 through 3 (of 3 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #13259

    tedr
    Member

    Hi Jeff,

    The HMC node is not a master or a slave in the same sense that hadoop has masters and slaves. The HMC node is the node on which the main HMC service is installed and run. I am looking into whether or not it can be installed without a datanode. It is possible to decommission the datanode that is installed with it if if cannot be installed without it.

    Thanks,
    Ted.

    Collapse
    #13238

    Hi Tedr:

    Is HMC node master or slave ?

    Can I deploy a cluster that HMC node with no data node ?

    Due to HMC node failure , I want my HMC node with HMC function only,

    but in fact , I found that HMC always configure a data node with itself.

    Is it a bug?

    Regards,
    Jeff

    Collapse
    #13223

    tedr
    Member

    Hi Jeff,

    You can separate the masters from the slaves, during the HMC install you have the ability to change what components go to which node. there is a step in the install that shows you which components are going to each node on this same step to the right of this list there are drop downs for each component you can select a different node from this list.

    Thanks.

    Collapse
Viewing 3 replies - 1 through 3 (of 3 total)