I am new to this forum and only recently did I start configuring my own cluster.
I am following the manual installation instructions available here:
I have 1 master and 2 slaves/data nodes running as CentOS virtual images, all of which have 100 GB disk, and 2 GB RAM.
I am stuck in the smoke test of mapreduce (5.4). I start the command to generate some random data, the job gets created and accepted, but remains unassigned. I looked into the Namenode information and I see 2 live nodes.
I have absolutely no clue where to start looking. I googled it up, some people seem to say that it could be related to configurations (if I configured some files to have too much memory, the task manager will keep aborting due to lack of resources), I added some lines to conf/mapred-site.xml, but no change:
Finally, here is what I see on my screen:
[client@master41 ~]$ /usr/lib/hadoop/bin/hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-184.108.40.206.1.3.0-563.jar teragen 10000 tmp/teragenout
14/07/22 16:20:27 INFO client.RMProxy: Connecting to ResourceManager at master41/192.168.100.41:8050
14/07/22 16:20:27 INFO terasort.TeraSort: Generating 10000 using 2
14/07/22 16:20:27 INFO mapreduce.JobSubmitter: number of splits:2
14/07/22 16:20:28 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1406038152966_0001
14/07/22 16:20:28 WARN mapred.YARNRunner: Usage of -Djava.library.path in mapreduce.admin.map.child.java.opts can cause programs to no longer function if hadoop native libraries are used. These values should be set as part of the LD_LIBRARY_PATH in the map JVM env using mapreduce.admin.user.env config settings.
14/07/22 16:20:28 WARN mapred.YARNRunner: Usage of -Djava.library.path in mapreduce.admin.reduce.child.java.opts can cause programs to no longer function if hadoop native libraries are used. These values should be set as part of the LD_LIBRARY_PATH in the reduce JVM env using mapreduce.admin.user.env config settings.
14/07/22 16:20:28 INFO impl.YarnClientImpl: Submitted application application_1406038152966_0001
14/07/22 16:20:28 INFO mapreduce.Job: The url to track the job: http://master41:8088/proxy/application_1406038152966_0001/
14/07/22 16:20:28 INFO mapreduce.Job: Running job: job_1406038152966_0001
Any hints to where I should look at?