Home Forums YARN JobHistory Web can not view Log

Tagged: ,

This topic contains 3 replies, has 3 voices, and was last updated by  Diwakar Dhanuskodi 2 months ago.

  • Creator
    Topic
  • #57695

    Bk Choi
    Participant

    I setup conig the following: (Of course, I restarted YARN/MR related servers)
    - YARN: yarn.log-aggregation-enable = true
    - YARN: yarn.nodemanager.remote-app-log-dir = /app-logs

    And, after some MR completed, the following command showed log of MR Job application.

    # mapred job -logs job_1405996066644_0010

    But, JobHistory UI(http://bt4:19888/jobhistory) did not show job log, instead following message.

    Aggregation is not enabled. Try the nodemanager at bt2:45454

    How can I see job log in JobHistory UI?
    Do I need some addiotional config modification?

Viewing 3 replies - 1 through 3 (of 3 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #57783

    Diwakar Dhanuskodi
    Participant

    You will be able to see the logs when you use resource manager UI and it does have link to history logs for the completed jobs. History logs will be created for a application jobs in YARN only when the jobs are completed.

    Collapse
    #57698

    Bk Choi
    Participant

    Yes, I installed HDP by using Ambari.

    jps command is following:

    [root@bt4 bin]# ./jps
    12562 ApplicationHistoryServer
    469 NameNode
    6661 SecondaryNameNode

    21106 AmbariServer
    29451 JobHistoryServer
    12863 ResourceManager

    I saw a JobHistoryServer process, and Ambari web told me that history server is running(started).

    There are some files in HDFS /app-logs/hdfs/logs/application_1405996066644_0010 path.

    Does this means that MR logs are collected?

    Why JobHistory UI did not show job log?
    Why ‘mapred -log command’ show job log?

    Collapse
    #57696

    Rohit Bakhshi
    Moderator

    Hi Bk,

    Can you check to see if the JobHistoryServer is running? You can use the Java ‘jps’ tool on the node that hosts the JobHistoryServer to find out if it is running.

    Did you install this cluster via RPMs or via Ambari?

    Collapse
Viewing 3 replies - 1 through 3 (of 3 total)