Home Forums Pig How to create ordered values?

This topic contains 2 replies, has 2 voices, and was last updated by  Pawel Kowalski 1 year, 1 month ago.

  • Creator
    Topic
  • #40776

    Pawel Kowalski
    Participant

    Hi
    I have logs like this:
    14:22:09,835| ERROR | 81258617B015E6BE9KOONREB35886539.ec3www04 | au2 | 400
    14:22:09,844| DEBUG | 81258617B015E6BE9KOONREB35886539.ec3www04 | au1 | 0
    14:26:03,438| INFO | 81258617B015E6BE9C9383EB35889313.ec3www04 | au1 | 1000
    14:26:03,444| DEBUG | 5A3BB0439CB862F719866262A0868AC3.ec3www04 | au1 | 1000
    14:59:31,054| INFO | 81258617B015E6BE9C9383EB35889313.ec3www04 | au1 | 1090
    14:59:45,518| INFO | 81258617B015E6BE9C9383EB35889313.ec3www04 | au2 | 1100
    15:01:29,583| INFO | 81258617B015E6BE9C9383EB35889313.ec3www04 | au2 | 1701
    15:01:30,449| DEBUG | 5A3BB0439CB862F719866262A0868AC3.ec3www04 | au1 | 1010

    and fields are date, log_level, session_id, user_id, event_code.
    I need to group events by session_id, user_id and present list of event_code orderd by date.

    file = LOAD ‘logs.log’ using PigStorage(‘|’)
    AS (date : chararray, log_level : chararray, session_id : chararray, user_id : chararray, event_code : int);
    session = group events by (session_id, user_id);

    and what next? How do i sort it?

    Please help.
    Thanks
    Pawel

Viewing 2 replies - 1 through 2 (of 2 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #40864

    Pawel Kowalski
    Participant

    (…)
    2013-10-17 11:18:31,071 [main] ERROR org.apache.pig.tools.grunt.Grunt – ERROR 1200: Pig script failed to parse:
    expression is not a project expression: (Name: ScalarExpression) Type: null Uid: null)
    Details at logfile: /hadoop/mapred/taskTracker/hue/jobcache/job_201310161143_0042/attempt_201310161143_0042_m_000000_0/work/pig_1382033897387.log

    and line 6 is {ordered = order file by date;

    any idea?

    Collapse
    #40778

    Jianyong Dai
    Participant

    result = foreach session {ordered = order file by date; generate ordered;};

    Collapse
Viewing 2 replies - 1 through 2 (of 2 total)