Error while running MR program in python

to create new topics or reply. | New User Registration


This topic contains 2 replies, has 2 voices, and was last updated by  Seth Lyubich 1 year, 8 months ago.

  • Creator
  • #43033

    Sorna Lingam

    HI all

    I have installed hadoop 1.3 msi on my machine and run the following MR program

    hadoop jar /HDP/hadoop- \ -mapper “python C:\Python33\” \ -reducer “python C:\Python33\” \ -input “C:\Python33\mm.txt” \ -output “C:\Python33\out.txt”

    but im getting the following error

    2013-11-06 11:13:12,933 WARN No groups available for user sornalingam
    2013-11-06 11:13:13,177 INFO org.apache.hadoop.mapred.JobClient: Cleaning up the staging area hdfs://DEV144:8020/mapred/staging/sornalingam/.staging/job_201311061035_0010

    can any one help me what went wrong



Viewing 2 replies - 1 through 2 (of 2 total)

You must be to reply to this topic. | Create Account

  • Author
  • #43592

    Seth Lyubich

    Hi Sorna,

    Can you please make sure that you can execute other jobs as user sornalingam. Can you also try by specifying full hdfs path -hdfs://……//user/sornalingam/input/mm.txt, or testing on local file system?

    Hope this helps,



    Sorna Lingam


    I have updated the code

    hadoop jar /HDP/hadoop- \ -mapper “python C:\Python33\map.cmd” \ -reducer “python C:\Python33\reducer.cmd” \ -input “/user/sornalingam/input/mm.txt” \ -output “/user/sornalingam/output”

    and my log containg

    2013-11-07 11:17:48,872 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop library
    2013-11-07 11:17:48,873 WARN Snappy native library not loaded
    2013-11-07 11:17:48,881 INFO org.apache.hadoop.mapred.FileInputFormat: Total input paths to process : 5
    2013-11-07 11:17:48,881 INFO org.apache.hadoop.mapred.JobClient: Cleaning up the staging area hdfs://DEV144:8020/mapred/staging/sornalingam/.staging/job_201311071036_0004
    2013-11-07 11:17:48,882 ERROR PriviledgedActionException as:sornalingam Not a file: hdfs://DEV144:8020/apps
    2013-11-07 11:17:48,883 ERROR org.apache.hadoop.streaming.StreamJob: Error Launching job : Not a file: hdfs://DEV144:8020/apps

    what went wrong

Viewing 2 replies - 1 through 2 (of 2 total)
Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.