HDP on Linux – Installation Forum

How to recover unhealthy node

  • #49510
    Peter Teunissen

    I’m installing HDP2.0 on Ubuntu 12.04 as a single node test setup. I got to the point where I’ve got HDFS and YARN installed and running. However the single datanode I’ve got shown as unhealthy in the YARN webapp. The message is 1/1 local-dirs turned bad: /grid/hadoop/hdfs/yarn/local;
    I noticed that I didn’t create the directory. So I created it and updated ownership and access rights. Even after reboots and restarts of the different daemons, the node is still shown as unhealthy.

    What additional things do I need to do to recover the datanode and get it ‘healthy’ again?

to create new topics or reply. | New User Registration

  • Author
  • #50440
    Ryan Suarez

    Same problem. Anyone?


    Hi Peter,

    On this datanode can you run ls -al on /grid/hadoop/hdfs/yarn/local and print it here?



    Peter Teunissen


    root@hadoop:/# ls -al /grid/hadoop/hdfs/yarn/local
    total 8,0K
    drwxr-xr-x 2 hdfs hadoop 4,0K mrt  3 15:47 .
    drwxr-xr-x 3 hdfs hadoop 4,0K mrt  3 15:47 ..

    Hi Peter,

    This directory should be owned by yarn:hadoop
    Can you change this directory and let me know if the problem persists?



    Peter Teunissen

    That solved the issue! After restarting yarn, the node is shown as healthy now.


    Andrei Bogdan

    @Peter Teunissen

    Could you please explain step-by-step what did you do to restart yarn? What commands ?
    I am having an unhealthy node and I cannot use my Hadoop at all. The only solution for me now is to install a new Hadoop distribution.

    Thank you in advance!

    Peter Teunissen


    It’s a long time ago and I can’t finde my notes so I can’t give you my step-by-step procedure. But, look at https://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-site/YarnCommands.html. There you’ll find the yarn commands. I think you need something like

    yarn application -list

    to get the application ID’s, then

    yarn aplication -kill <ID>

    . To start the applications simply do:

    yarn resourcemanager


    yarn nodemanager


    yarn proxyserver



You must be to reply to this topic. | Create Account

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.