Home Forums HDP on Windows – Installation libhdfs and java_library_path

This topic contains 0 replies, has 1 voice, and was last updated by  Stephen Bovy 10 months ago.

  • Creator
    Topic
  • #44435

    Stephen Bovy
    Member

    To setup for libhdfs and pass in the java_library_path we need to update “hadoop-config.cmd” and “hadoop.cmd” as follows

    hadoop-config.cmd::

    @rem For the disto case, check the lib\native folder
    if exist %HADOOP_CORE_HOME%\lib\native (

    if defined JAVA_LIBRARY_PATH (
    set JAVA_LIBRARY_PATH=%JAVA_LIBRARY_PATH%;%HADOOP_CORE_HOME%\lib\native\%JAVA_PLATFORM%;%HADOOP_CORE_HOME%\lib\native
    ) else (
    set JAVA_LIBRARY_PATH=%HADOOP_CORE_HOME%\lib\native\%JAVA_PLATFORM%;%HADOOP_CORE_HOME%\lib\native
    )

    )

    if defined JAVA_LIBRARY_PATH (
    set HADOOP_OPTS=%HADOOP_OPTS% -Djava.library.path=%JAVA_LIBRARY_PATH%
    set LIBHDFS_OPTS=-Djava.library.path=%JAVA_LIBRARY_PATH%
    )

    hadoop.cmd ::
    :print_usage
    @echo Usage: hadoop [--config confdir] COMMAND
    @echo where COMMAND is one of:
    @echo namenode -format format the DFS filesystem
    @echo secondarynamenode run the DFS secondary namenode
    @echo namenode run the DFS namenode
    @echo datanode run a DFS datanode
    @echo dfsadmin run a DFS admin client
    @echo mradmin run a Map-Reduce admin client
    @echo fsck run a DFS filesystem checking utility
    @echo fs run a generic filesystem user client
    @echo balancer run a cluster balancing utility
    @echo snapshotDiff diff two snapshots of a directory or diff the
    @echo current directory contents with a snapshot
    @echo lsSnapshottableDir list all snapshottable dirs owned by the current user
    @echo oiv apply the offline fsimage viewer to an fsimage
    @echo fetchdt fetch a delegation token from the NameNode
    @echo jobtracker run the MapReduce job Tracker node
    @echo pipes run a Pipes job
    @echo tasktracker run a MapReduce task Tracker node
    @echo historyserver run job history servers as a standalone daemon
    @echo job manipulate MapReduce jobs
    @echo queue get information regarding JobQueues
    @echo version print the version
    @echo jar ^ run a jar file
    @echo.
    @echo distcp ^ ^ copy file or directories recursively
    @echo distcp2 ^ ^ DistCp version 2
    @echo archive -archiveName NAME ^* ^ create a hadoop archive
    @echo daemonlog get/set the log level for each daemon
    @echo or
    @echo CLASSNAME run the class named CLASSNAME
    @echo Most commands print help when invoked w/o parameters.

    rem export variables for libhdfs

    @echo %JAVA_LIBRARY_PATH% > myclassxx
    @echo %LIBHDFS_OPTS% > libhdfs

    rem export variables for libhdfs

    endlocal

    rem export variables for libhdfs

    for /F “usebackq” %%i IN (`type myclassxx`) do set JAVA_LIBRARY_PATH=%%i
    @del myclassxx

    for /F “usebackq” %%i IN (`type libhdfs`) do set LIBHDFS_OPTS=%%i
    @del libhdfs

You must be logged in to reply to this topic.