Cannot see MR logs in Hadoop 2.x

to create new topics or reply. | New User Registration

This topic contains 3 replies, has 3 voices, and was last updated by  Anand M 1 year, 2 months ago.

  • Creator
  • #46485

    Kyle Lin

    My Environment: HDP 2 (Hadoop 2.2.0).

    When using Hadoop 1.x, I write log imformation in my MapReduce code, then see the log message in TaskTracker log. But In YARN, I cannot find the log message in any log files(historyserver, nodemanager, resourcemanager), even using System.out.println.

    I also try to see app log through the address of yarn.resourcemanager.webapp.address. Still cannot see my log messages.

    Where can I find the log files with my log message?

Viewing 3 replies - 1 through 3 (of 3 total)

You must be to reply to this topic. | Create Account

  • Author
  • #54415

    Anand M

    A MR job has a unique application id in the format – “application_<application id>_<job sequence no>”

    After the completion of your job, execute yarn command as below to get your logs.

    yarn logs “application_<application id>_<job sequence no>” > /store_it_somewhere_in_your_local_logs_path

    Please read this link for details.



    D Blair Elzinga

    You *should* be able to go the history server web page and view the jobs: <historyserver>:19888/jobhistory.

    Or from the resource manager <resource manager host>:8088/cluster to view all applications, then click on the ID link, then on “logs” link.


    Kyle Lin

    Sorry for my rough asking. Refine my question.

    When using hadoop 1.x, I can see the MR job logs under “userlogs” folder in local file system. But I cannot find the MR log in hadoop 2.x environment. There is no “userlogs” folder.

    Can anybody tell me where to find the log files?

Viewing 3 replies - 1 through 3 (of 3 total)
Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.