The Hortonworks Community Connection is now live. A completely rebuilt Q&A forum, Knowledge Base, Code Hub and more, backed by the experts in the industry.

You will be redirected here in 10 seconds. If your are not redirected, click here to visit the new site.

The legacy Hortonworks Forum is now closed. You can view a read-only version of the former site by clicking here. The site will be taken offline on January 31,2016

HDFS Forum

Decomission Data Node / Redistribute data among remaining data node

  • #49620
    Gwenael Le Barzic

    Hello !

    I have a very small cluster of 4 nodes :
    – 1 master on which were also installed a data node
    – 3 slaves (on which there is data node)

    After a short training around Big Data technology, I realized it was better to remove the data node from the master, in order to release some resources on the master.

    So, I went into the ambari user interface dashboard, and then in the master node, in the services installed on this master node, and, for the data node, I performed a “decomission”.

    And now this data node is just blinking in orange.

    So I contact you because I’m not sure about what to do now.
    1. How can I request a new distribution of the data, now this data node is decomissioned
    2. Will this data node completely disappear from ambari ? Do I have to do something more ?

    Best regards.


  • Author
  • #50911
    Koelli Mungee

    Hi Gwenael,

    Based on your description it seems like you had a total of 3 datanodes, is that accurate or did you have 4 datanodes. If you had 3 datanodes, then ideally in a 3 node cluster the decommission would not go through since the replication factor is 3. Can you check the namenode UI to see the state of the datanode that was decommissioned? This could help shed some light on the scenario.


    Gwenael Le Barzic

    Hello !

    Hm, let me try to clarify my situation :
    Originally, I had one master node and 3 data nodes.
    But, on the master node, we installed also a data node, and I think it was a bad idea.

    So I tried to decomission the data module from the master node.
    And here I am.

    Is it more clear ?

    Best regards.

    Gwenael Le Barzic

    Koelli Mungee

    Hi Gwenael,

    Thanks for the clarification! It sounds like you should add a new datanode and then decommision the datanode that is on the master node. That way, you can ensure that the blocks always have a replication factor of 3. Also makes sure there aren’t any files that have a higher replication factor on the HDFS.

    Hope this helps

The forum ‘HDFS’ is closed to new topics and replies.

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.