HOWTO: Test MapReduce Setup

ISSUE

How do I run an example map reduce job? Or

How do I test the map reduce services are working?

SOLUTION

Make sure the job tracker and the task trackers are started.

To start the job tracker:

su mapred - -c "hadoop-daemon.sh --config /etc/hadoop start jobtracker; sleep 25"

To start a task tracker:

su mapred - -c "hadoop-daemon.sh --config /etc/hadoop start tasktracker"

Run a map reduce job from the hadoop examples jar. This jar packages up a few example map reduce classes. The following command runs the sleep example with one mapper and one reducer:

hadoop jar /usr/share/hadoop/hadoop-examples-1.0.0.jar sleep -m 1 -r 1

The map reduce job will write outpts to the console. These outputs provide the job id that can be used to track the status of the job. The console output also displays the progress of the maps and reducers.

Thank you for subscribing!