The Hortonworks Community Connection is now live. A completely rebuilt Q&A forum, Knowledge Base, Code Hub and more, backed by the experts in the industry.

You will be redirected here in 10 seconds. If your are not redirected, click here to visit the new site.

The legacy Hortonworks Forum is now closed. You can view a read-only version of the former site by clicking here. The site will be taken offline on January 31,2016

MapReduce Forum

MapR Job is not starting

  • #52351
    Lars
    Participant

    I am really new in Hadoop. I have installed HDP (manually) but MapR Job never starts:
    E.g.:
    hdfs@test.apo.lan:/home/$ /usr/lib/hadoop/bin/hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.2.0.2.0.11.0-1.jar teragen 10000 /tmp/teragenout

    Job seems to be started but never starts.

    14/04/28 12:01:47 INFO impl.YarnClientImpl: Submitted application application_1398339026838_0005 to ResourceManager at kvm.apo.lan/192.168.235.139:8050
    14/04/28 12:01:47 INFO mapreduce.Job: The url to track the job: http://test.apo.lan:8088/proxy/application_1398339026838_0005/
    14/04/28 12:01:47 INFO mapreduce.Job: Running job: job_1398339026838_0005

    14/04/28 11:59:23 INFO hs.JobHistoryServer: STARTUP_MSG:
    /************************************************************
    STARTUP_MSG: Starting JobHistoryServer
    STARTUP_MSG:   host = test.apo.lan/192.168.235.139
    STARTUP_MSG:   args = []
    STARTUP_MSG:   version = 2.2.0.2.0.11.0-1
    STARTUP_MSG:   classpath = /etc/hadoop/conf:/usr/lib/hadoop/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop/lib/netty-3.6.2.Final.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/l$
    STARTUP_MSG:   build = git@github.com:hortonworks/hadoop.git -r a226d56a6ec93da79a316305f92d156ec0c2a7d6; compiled by 'jenkins' on 2014-03-12T09:49Z
    STARTUP_MSG:   java = 1.7.0_51
    ************************************************************/
    14/04/28 11:59:23 INFO hs.JobHistoryServer: registered UNIX signal handlers for [TERM, HUP, INT]
    14/04/28 11:59:24 INFO hs.JobHistory: JobHistory Init
    14/04/28 11:59:25 INFO hs.HistoryFileManager: Initializing Existing Jobs...
    14/04/28 11:59:25 INFO hs.CachedHistoryStorage: CachedHistoryStorage Init
    14/04/28 11:59:25 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
    14/04/28 11:59:25 INFO impl.MetricsSystemImpl: Scheduled snapshot period at 60 second(s).
    14/04/28 11:59:25 INFO impl.MetricsSystemImpl: JobHistoryServer metrics system started
    14/04/28 11:59:25 INFO delegation.AbstractDelegationTokenSecretManager: Updating the current master key for generating delegation tokens
    14/04/28 11:59:25 INFO delegation.AbstractDelegationTokenSecretManager: Starting expired delegation token remover thread, tokenRemoverScanInterval=60 
    ...
    14/04/28 11:59:26 INFO ipc.Server: IPC Server Responder: starting
    14/04/28 11:59:26 INFO ipc.Server: IPC Server listener on 10020: starting
    14/04/28 11:59:26 INFO hs.HistoryClientService: Instantiated MRClientService at test.apo.lan/192.168.235.139:10020
    14/04/28 11:59:26 INFO logaggregation.AggregatedLogDeletionService: aggregated log deletion started.
    14/04/28 11:59:26 INFO logaggregation.AggregatedLogDeletionService: aggregated log deletion finished.
    14/04/28 11:59:55 INFO hs.JobHistory: History Cleaner started
    14/04/28 11:59:55 INFO hs.JobHistory: History Cleaner complete
    14/04/28 12:02:25 INFO hs.JobHistory: Starting scan to move intermediate done files

    But nothing starts. Please help!

  • Author
    Replies
  • #52608
    Koelli Mungee
    Moderator

    Hi Lars,

    Can you please check the Resource Manager UI to track the job, do you see it there?

    Regards
    Koelli

    #53168
    Lars
    Participant

    No, It shows:

    Apps Submitted: 6
    Apps Pending : 6
    Unhealthy Nodes: 1 (because its a pseudy-cluster, running for evaluation on one host)

    Furthermore (the last job):
    ID: application_1398339026838_0001
    User: client
    Name: TeraGen
    Application Type: MAPREDUCE
    Queue: default
    StartTime: Thu, 24 Apr 2014 11:33:04 GMT
    FinishTime: N/A
    State: ACCEPTED
    FinalStatus: UNDIFENED

    Any ideas where i have to look?

The forum ‘MapReduce’ is closed to new topics and replies.

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.