Home Forums HDFS Decomission Data Node / Redistribute data among remaining data node

This topic contains 3 replies, has 2 voices, and was last updated by  Koelli Mungee 8 months ago.

  • Creator
    Topic
  • #49620

    Gwenael Le Barzic
    Participant

    Hello !

    I have a very small cluster of 4 nodes :
    – 1 master on which were also installed a data node
    – 3 slaves (on which there is data node)

    After a short training around Big Data technology, I realized it was better to remove the data node from the master, in order to release some resources on the master.

    So, I went into the ambari user interface dashboard, and then in the master node, in the services installed on this master node, and, for the data node, I performed a “decomission”.

    And now this data node is just blinking in orange.

    So I contact you because I’m not sure about what to do now.
    1. How can I request a new distribution of the data, now this data node is decomissioned
    2. Will this data node completely disappear from ambari ? Do I have to do something more ?

    Best regards.

    Gwenael

Viewing 3 replies - 1 through 3 (of 3 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #51570

    Koelli Mungee
    Moderator

    Hi Gwenael,

    Thanks for the clarification! It sounds like you should add a new datanode and then decommision the datanode that is on the master node. That way, you can ensure that the blocks always have a replication factor of 3. Also makes sure there aren’t any files that have a higher replication factor on the HDFS.

    Hope this helps
    -koelli

    Collapse
    #51131

    Gwenael Le Barzic
    Participant

    Hello !

    Hm, let me try to clarify my situation :
    Originally, I had one master node and 3 data nodes.
    But, on the master node, we installed also a data node, and I think it was a bad idea.

    So I tried to decomission the data module from the master node.
    And here I am.

    Is it more clear ?

    Best regards.

    Gwenael Le Barzic

    Collapse
    #50911

    Koelli Mungee
    Moderator

    Hi Gwenael,

    Based on your description it seems like you had a total of 3 datanodes, is that accurate or did you have 4 datanodes. If you had 3 datanodes, then ideally in a 3 node cluster the decommission would not go through since the replication factor is 3. Can you check the namenode UI to see the state of the datanode that was decommissioned? This could help shed some light on the scenario.

    thanks
    koelli

    Collapse
Viewing 3 replies - 1 through 3 (of 3 total)