YARN Forum

JobHistory Web can not view Log

  • #57695
    Bk Choi

    I setup conig the following: (Of course, I restarted YARN/MR related servers)
    – YARN: yarn.log-aggregation-enable = true
    – YARN: yarn.nodemanager.remote-app-log-dir = /app-logs

    And, after some MR completed, the following command showed log of MR Job application.

    # mapred job -logs job_1405996066644_0010

    But, JobHistory UI(http://bt4:19888/jobhistory) did not show job log, instead following message.

    Aggregation is not enabled. Try the nodemanager at bt2:45454

    How can I see job log in JobHistory UI?
    Do I need some addiotional config modification?

to create new topics or reply. | New User Registration

  • Author
  • #57696
    Rohit Bakhshi

    Hi Bk,

    Can you check to see if the JobHistoryServer is running? You can use the Java ‘jps’ tool on the node that hosts the JobHistoryServer to find out if it is running.

    Did you install this cluster via RPMs or via Ambari?

    Bk Choi

    Yes, I installed HDP by using Ambari.

    jps command is following:

    [root@bt4 bin]# ./jps
    12562 ApplicationHistoryServer
    469 NameNode
    6661 SecondaryNameNode

    21106 AmbariServer
    29451 JobHistoryServer
    12863 ResourceManager

    I saw a JobHistoryServer process, and Ambari web told me that history server is running(started).

    There are some files in HDFS /app-logs/hdfs/logs/application_1405996066644_0010 path.

    Does this means that MR logs are collected?

    Why JobHistory UI did not show job log?
    Why ‘mapred -log command’ show job log?

    Diwakar Dhanuskodi

    You will be able to see the logs when you use resource manager UI and it does have link to history logs for the completed jobs. History logs will be created for a application jobs in YARN only when the jobs are completed.

You must be to reply to this topic. | Create Account

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.