JobHistory Web can not view Log

to create new topics or reply. | New User Registration

Tagged: ,

This topic contains 3 replies, has 3 voices, and was last updated by  Diwakar Dhanuskodi 8 months, 1 week ago.

  • Creator
  • #57695

    Bk Choi

    I setup conig the following: (Of course, I restarted YARN/MR related servers)
    – YARN: yarn.log-aggregation-enable = true
    – YARN: yarn.nodemanager.remote-app-log-dir = /app-logs

    And, after some MR completed, the following command showed log of MR Job application.

    # mapred job -logs job_1405996066644_0010

    But, JobHistory UI(http://bt4:19888/jobhistory) did not show job log, instead following message.

    Aggregation is not enabled. Try the nodemanager at bt2:45454

    How can I see job log in JobHistory UI?
    Do I need some addiotional config modification?

Viewing 3 replies - 1 through 3 (of 3 total)

You must be to reply to this topic. | Create Account

  • Author
  • #57783

    Diwakar Dhanuskodi

    You will be able to see the logs when you use resource manager UI and it does have link to history logs for the completed jobs. History logs will be created for a application jobs in YARN only when the jobs are completed.


    Bk Choi

    Yes, I installed HDP by using Ambari.

    jps command is following:

    [root@bt4 bin]# ./jps
    12562 ApplicationHistoryServer
    469 NameNode
    6661 SecondaryNameNode

    21106 AmbariServer
    29451 JobHistoryServer
    12863 ResourceManager

    I saw a JobHistoryServer process, and Ambari web told me that history server is running(started).

    There are some files in HDFS /app-logs/hdfs/logs/application_1405996066644_0010 path.

    Does this means that MR logs are collected?

    Why JobHistory UI did not show job log?
    Why ‘mapred -log command’ show job log?


    Rohit Bakhshi

    Hi Bk,

    Can you check to see if the JobHistoryServer is running? You can use the Java ‘jps’ tool on the node that hosts the JobHistoryServer to find out if it is running.

    Did you install this cluster via RPMs or via Ambari?

Viewing 3 replies - 1 through 3 (of 3 total)
Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.