The Hortonworks Community Connection is now live. A completely rebuilt Q&A forum, Knowledge Base, Code Hub and more, backed by the experts in the industry.

You will be redirected here in 10 seconds. If your are not redirected, click here to visit the new site.

The legacy Hortonworks Forum is now closed. You can view a read-only version of the former site by clicking here. The site will be taken offline on January 31,2016

HDP on Linux – Installation Forum

hadoop format formats wrong directory

  • #14082

    when trying to perform a manual installation I have come across a weird behavior when performing the command ‘su – hdfs -c “/usr/lib/hadoop/bin/hadoop namenode -format” ‘.
    Hadoop tries to format a tmp folder (/tmp/hadoop-${}), which is not the folder specified in hdfs-site.xml under the /
    As a result, starting the namenode after the format fails since the namenode is not located in the formatted location.

    Investigating this subject shows that the value for the tmp folder is taken from a variable hadoop.tmp.dir, but I can’t find it in hdfs-site.xml or in core-site.xml.
    Am I missing something, how Can I make sure hadoop namenode -format uses the correct variable from hdfs-site.xml?


  • Author
  • #14089
    Larry Liu

    Hi, Tzur

    Thanks fro trying HDP.

    The property hadoop.tmp.dir is a default property for hadoop. It can be overwriten in core-site.xml. By the way, are you trying to install HDP 1.2?



    Hello Larry,
    I’m trying to install Hadoop 1.2 manually (RPMs) after the Amabari installation failed.

    I don’t have that property in core-site.xml, but I can add it manually.
    the question is why do I even need to specify it. Why the format option get an invalid data directory and does not use the one specified in hdfs-site.xml in the propety


    I added the property hadoop.tmp.dir to hdfs-site.xml and re-formatted hdfs. It is still ignoring the property and tries to format:

    when my data/name dirs are in

    I’m confused :-\

    Larry Liu

    Hi, Tzur,

    The hadoop.tmp.dir should be added into core-site.xml. Can you please try again.


    Larry Liu

    Hi, Tzur,

    Can you clarify if the system is clean when you install manually after ambari installation failed? It is better to start the installation from a clean system.



    Hi Larry,
    I created new VMs for non-ambari installation. Easier than cleaning a system 😉

    Ok, so I moved the property from hdfs-site.xml to core-site.xml,
    but when trying to format I’m still getting this:
    Re-format filesystem in /tmp/hadoop-hdfs/dfs/name ? (Y or N)

    when the property in core-site.xml is:
    name hadoop.tmp.dir name
    value /mnt/hadoop/dataNode value

    Larry Liu

    Hi, Tzur

    Can you please provide the namenode log file and upload according to the following link?




    Hi Larry, produces an empty …out file
    There are also errors on /var/db/hmc/data.db which does not exist.

    I appreciate your help, but luckily I am working with some of Hortonworks employees on a mutual project and I’ll try and get their onsite help on this issue.

    Thank you for your time!

    Larry Liu

    Hi, Tzur

    The is for HDP 1.1. I guess it doesn’t work with HDP 1.2.

    Can you please get namenode log only for me?



    Larry Liu

    Hi, Tzur

    Just want to follow up with you the updated status of your installation.


The forum ‘HDP on Linux – Installation’ is closed to new topics and replies.

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.