HOWTO: Test HDFS Setup

ISSUE

How do I run simple Hadoop Distributed File System tasks? Or

How do I test that HDFS services are working?

SOLUTION

Make sure the name node and the data nodes are started.

To start the name node:

su hdfs - -c "hadoop-daemon.sh --config /etc/hadoop/ start namenode"

To start a data node:

su hdfs - -c "hadoop-daemon.sh --config /etc/hadoop start datanode"

Put data files into HDFS. This command will take a file from disk and put into HDFS:

su hdfs
hadoop fs -put trial_file.csv /user/hdfs/trial_file.csv

Read data from HDFS. This command will read the contents of a file from HDFS and display on the console:

su hdfs
hadoop fs -cat /user/hdfs/trial_file.csv

References:

http://hadoop.apache.org/common/docs/current/file_system_shell.html

Try these Tutorials

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.