Home Forums MapReduce Cannot see MR logs in Hadoop 2.x

This topic contains 3 replies, has 3 voices, and was last updated by  Anand M 6 months, 4 weeks ago.

  • Creator
  • #46485

    Kyle Lin

    My Environment: HDP 2 (Hadoop 2.2.0).

    When using Hadoop 1.x, I write log imformation in my MapReduce code, then see the log message in TaskTracker log. But In YARN, I cannot find the log message in any log files(historyserver, nodemanager, resourcemanager), even using System.out.println.

    I also try to see app log through the address of yarn.resourcemanager.webapp.address. Still cannot see my log messages.

    Where can I find the log files with my log message?

Viewing 3 replies - 1 through 3 (of 3 total)

You must be logged in to reply to this topic.

  • Author
  • #54415

    Anand M

    A MR job has a unique application id in the format – “application_<application id>_<job sequence no>”

    After the completion of your job, execute yarn command as below to get your logs.

    yarn logs “application_<application id>_<job sequence no>” > /store_it_somewhere_in_your_local_logs_path

    Please read this link for details.




    D Blair Elzinga

    You *should* be able to go the history server web page and view the jobs: <historyserver>:19888/jobhistory.

    Or from the resource manager <resource manager host>:8088/cluster to view all applications, then click on the ID link, then on “logs” link.


    Kyle Lin

    Sorry for my rough asking. Refine my question.

    When using hadoop 1.x, I can see the MR job logs under “userlogs” folder in local file system. But I cannot find the MR log in hadoop 2.x environment. There is no “userlogs” folder.

    Can anybody tell me where to find the log files?

Viewing 3 replies - 1 through 3 (of 3 total)