Home Forums HDP on Windows – Installation Hadoop configuration files issues

This topic contains 1 reply, has 1 voice, and was last updated by  Raja Nagendra Kumar 5 months, 1 week ago.

  • Creator
    Topic
  • #51342

    Raja Nagendra Kumar
    Participant

    All the configuration files generated by install on windows are resulting in URI being wrong.

    e.g
    core-site.xml

    <property>
    <name>hadoop.tmp.dir</name>
    <value>file:///D:/Apps/java/BI/hadoop/hw/hdpdata/hdfs/tmp</value>
    </property>

    i.e prefix file:/// and change \ to / in dir

    similarly

    in mapred.xml

    <property>
    <name>mapred.child.tmp</name>
    <value>file:///D:/Apps/java/BI/hadoop/hw/hdpdata/hdfs/tmp</value>
    </property>

    <property>
    <name>mapred.local.dir</name>
    <value>file:///D:/Apps/java/BI/hadoop/hw/hdpdata/hdfs/mapred/local</value>
    </property>

    <property>
    <name>mapreduce.cluster.local.dir</name>
    <value>file:///D:/Apps/java/BI/hadoop/hw/hdpdata/hadoop/local</value>
    </property>

    in YARN.xml

    <property>
    <name>yarn.nodemanager.log-dirs</name>
    <value>file:///D:/Apps/java/BI/hadoop/hw/hdpdata/hadoop/logs</value>
    </property>

    <property>
    <name>yarn.nodemanager.local-dirs</name>
    <value>file:///D:/Apps/java/BI/hadoop/hw/hdpdata/hadoop/local</value>
    </property>

    in
    <property>
    <name>dfs.namenode.name.dir</name>
    <value>file:///D:/Apps/java/BI/hadoop/hw/hdpdata/hdfs/nn</value>
    <description>Determines where on the local filesystem the DFS name node
    should store the name table. If this is a comma-delimited list
    of directories then the name table is replicated in all of the
    directories, for redundancy. </description>
    <final>true</final>
    </property>

Viewing 1 replies (of 1 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #51456

    Raja Nagendra Kumar
    Participant

    With the above changes i.e change all the files to file:///xyz with unix style directory separator, HDP for windows works like a charm on Windows 7 too.
    Some more observations were
    if you have multiple network cards, then namenode is picking up some ip, than datanode. So use browser URL for namenode as one of the IP’s you have.
    Combine this install with http://bigdata.red-gate.com/ explorer install, would make you forget about HDFS commands.

    Working on HDF windows is awesome.

    Collapse
Viewing 1 replies (of 1 total)