MapReduce Forum

how do I complie my java program?

  • #46575
    Jason Cao

    hi, there
    I want to compile my mapreduce program, using javac, but for -classpath, I don’t know how to tell it where the hadoop jar files are.
    there are simply so many jar files put in different directories. how do I run the javac command?

to create new topics or reply. | New User Registration

  • Author
  • #46580

    Hi Jason Cao,
    Step 1) You have to make a .jar file of your right click on your project then export in jar
    Step 2) go to command line and copy your input file into hadoop hdfs by this command
    hadoop dfs -copyFromLocal /home/cloudera/yourfilename /

    Step 3) You can see your input file in hdfs by this command
    hadoop dfs -ls /

    Step 4) to execute your program use this
    hadoop jar yourjarfile.jar classname /InputFileName /OutputFileName

    Step 5) See your output file by
    hadoop fs -ls /OutputFileName

    Step 6)See your output data on commandline by
    hadoop dfs -cat /OutputFilename/part-r-00000

    Kishore Kumar Suthar
    BI Consultant
    linkdin Kishore Kumar Suthar

    Jason Cao

    hi Kishore,

    thanks for you reply, but sorry I may not stated my question clearly. I mean how to tell javac where those hadoop JAR files are when I try to complie my jave code.
    when I used hadoop 1.0, (I installed the cluster manually, not using horton or any rpm,) I can simply tell javac to use hadoop-core-xxx.jar which is under hadoop directry.
    but with the latest hortonworks cluster 2.0, I don’t know how to do that, there are tens of hadoop jar files scattered into different directories and I don’t know which one to use.
    Or, maybe I should define a CLASSPATH and put all jar files to this paramater, is this the suggested way?

    Dan Marshall

    Did you find out how to do this yet? I am baffled as well.

    Dan Marshall

    Run the “yarn classpath” command to see the required directories. I was able to compile my Java program after using the command output list in my -classpath.

You must be to reply to this topic. | Create Account

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.