Home Forums Hortonworks Sandbox Want to extract my server logs to HDFS

This topic contains 5 replies, has 3 voices, and was last updated by  Larry Liu 1 year, 8 months ago.

  • Creator
    Topic
  • #17647

    Mur Raguthu
    Member

    Is it Flume is the only solution? I was able to install Flume on Sandbox but I got sutck at configuration. Any help would be appreciated.

    Thanks.

Viewing 5 replies - 1 through 5 (of 5 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #18148

    Larry Liu
    Moderator

    Hi, Mur,

    I think the first thing to do is to set up flume agent on SQL servers.

    I am referring to the following web site:

    http://flume.apache.org/FlumeUserGuide.html#configuring-a-multi-agent-flow

    Please let us know when you have questions during configuration.

    Larry

    Collapse
    #18140

    Mur Raguthu
    Member

    Hi Larry,
    As of now we are able to pull messages from sandbox file system to HDFS (same server) via Flume-NG. We have multiple SQL servers and would like to pull those server logs to HDFS for analysis. That is the plan..

    Is that makes sense? Please let us know.

    Collapse
    #18105

    Larry Liu
    Moderator

    Hi, Mur,

    Can you please clarify what you are trying to do? From your last post, you are planning to extract other server logs in to Sandbox HDFS. I am wondering if you could provide more detail? We can start from here.

    Thanks
    Larry

    Collapse
    #17970

    Mur Raguthu
    Member

    We are able to set up Flume-NG in Sandbox and able to load ‘sandbox log files’ from same sandbox to HDFS. Now we are planning to extract other server logs in to Sandbox HDFS. Yes I am going through documentation and planning to set up like “$ telnet localhost 44444”. Did anybody done this with Sandbox? Share your experiences please?

    Collapse
    #17879

    tedr
    Moderator

    Hi Mur,

    Thanks for trying the Hortonworks Sandbox.
    Am I correct in assuming that you’ve read over the documentation at : http://flume.apache.org/FlumeUserGuide.html ?

    Thanks,
    Ted.

    Collapse
Viewing 5 replies - 1 through 5 (of 5 total)