HOWTO: Test MapReduce Setup

ISSUE

How do I run an example map reduce job? Or

How do I test the map reduce services are working?

SOLUTION

Make sure the job tracker and the task trackers are started.

To start the job tracker:

su mapred - -c "hadoop-daemon.sh --config /etc/hadoop start jobtracker; sleep 25"

To start a task tracker:

su mapred - -c "hadoop-daemon.sh --config /etc/hadoop start tasktracker"

Run a map reduce job from the hadoop examples jar. This jar packages up a few example map reduce classes. The following command runs the sleep example with one mapper and one reducer:

hadoop jar /usr/share/hadoop/hadoop-examples-1.0.0.jar sleep -m 1 -r 1

The map reduce job will write outpts to the console. These outputs provide the job id that can be used to track the status of the job. The console output also displays the progress of the maps and reducers.

Try these Tutorials

HDP 2.1 Webinar Series
Join us for a series of talks on some of the new enterprise functionality available in HDP 2.1 including data governance, security, operations and data access :
Contact Us
Hortonworks provides enterprise-grade support, services and training. Discuss how to leverage Hadoop in your business with our sales team.
Integrate with existing systems
Hortonworks maintains and works with an extensive partner ecosystem from broad enterprise platform vendors to specialized solutions and systems integrators.