Home Forums Hortonworks Sandbox How to run mapreduce job in sandbox?

Tagged: 

This topic contains 2 replies, has 3 voices, and was last updated by  Jeet Chatterjee 11 months, 2 weeks ago.

  • Creator
    Topic
  • #43705

    Krish N
    Member

    Thanks

Viewing 2 replies - 1 through 2 (of 2 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #46307

    i am trying to run a wordcount MR job in sanbox.but i am getting invalid jar error.This is my program

    public class WordCount {

    public static class Map extends MapReduceBase implements
    Mapper<LongWritable, Text, Text, IntWritable> {
    private final static IntWritable one = new IntWritable(1);
    private Text word = new Text();

    public void map(LongWritable key, Text value,
    OutputCollector<Text, IntWritable> output, Reporter reporter)
    throws IOException {
    String line = value.toString();
    StringTokenizer tokenizer = new StringTokenizer(line);
    while (tokenizer.hasMoreTokens()) {
    word.set(tokenizer.nextToken());
    output.collect(word, one);
    }
    }
    }

    public static class Reduce extends MapReduceBase implements
    Reducer<Text, IntWritable, Text, IntWritable> {
    public void reduce(Text key, Iterator<IntWritable> values,
    OutputCollector<Text, IntWritable> output, Reporter reporter)
    throws IOException {
    int sum = 0;
    while (values.hasNext()) {
    sum += values.next().get();
    }
    output.collect(key, new IntWritable(sum));
    }
    }

    public static void main(String[] args) throws IOException {
    JobConf conf = new JobConf(WordCount.class);
    conf.setJobName(“wordcount”);

    conf.setOutputKeyClass(Text.class);
    conf.setOutputValueClass(IntWritable.class);

    conf.setMapperClass(Map.class);
    conf.setCombinerClass(Reduce.class);
    conf.setReducerClass(Reduce.class);

    conf.setInputFormat(TextInputFormat.class);
    conf.setOutputFormat(TextOutputFormat.class);

    FileInputFormat.setInputPaths(conf, new Path(args[0]));
    FileOutputFormat.setOutputPath(conf, new Path(args[1]));

    JobClient.runJob(conf);

    }

    }

    i am trying to run in sandbox using hadoop jar C:\Users\Jeet\VirtualBox VMs\Hortonworks Sandbox 1.3\WordCount.jar

    but getting an invalid jar exception.C:\Users\Jeet\VirtualBox VMs\Hortonworks Sandbox 1.3 this is the path i have installed the sandbox.please help me..

    Collapse
    #43843

    Dave
    Moderator

    Hi Krish,

    You can follow the tutorials, or you can run a hive job which does a select from a table with a limit of 1000.

    This will start a MR job

    Thanks

    Dave

    Collapse
Viewing 2 replies - 1 through 2 (of 2 total)