HOWTO: Test HDFS Setup

ISSUE

How do I run simple Hadoop Distributed File System tasks? Or

How do I test that HDFS services are working?

SOLUTION

Make sure the name node and the data nodes are started.

To start the name node:

su hdfs - -c "hadoop-daemon.sh --config /etc/hadoop/ start namenode"

To start a data node:

su hdfs - -c "hadoop-daemon.sh --config /etc/hadoop start datanode"

Put data files into HDFS. This command will take a file from disk and put into HDFS:

su hdfs
hadoop fs -put trial_file.csv /user/hdfs/trial_file.csv

Read data from HDFS. This command will read the contents of a file from HDFS and display on the console:

su hdfs
hadoop fs -cat /user/hdfs/trial_file.csv

References:

http://hadoop.apache.org/common/docs/current/file_system_shell.html

Try these Tutorials

Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.
Integrate with existing systems
Hortonworks maintains and works with an extensive partner ecosystem from broad enterprise platform vendors to specialized solutions and systems integrators.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.