Home Forums Sqoop Dstributing work in Flume 1.4

Tagged: 

This topic contains 1 reply, has 2 voices, and was last updated by  Robert 1 year, 3 months ago.

  • Creator
    Topic
  • #30371

    I looked at Apache Flume 1.4 documentation and its not clear about how to distribute the work accross nodes. I have to fetch data from multiple sources and multiple query terms for each source and need to poll them regularly. Older versions of Flume for flume-master and flume-node(s). Cloudera release notes says these concepts are replaced by Agents. But how to distribute the agent(s)?
    Should I manually create multiple configurations on each node and start agent on each of the node? or can be done through any centralized mechanism?
    Thanks,

Viewing 1 replies (of 1 total)

The topic ‘Dstributing work in Flume 1.4’ is closed to new replies.

  • Author
    Replies
  • #30504

    Robert
    Participant

    Hi Maruti,
    The new generation of flume does not have a centralized mechanism. The agents are independent. You would have to launch each agent on each of the nodes.

    Regards,
    Robert

    Collapse
Viewing 1 replies (of 1 total)