Can twitter Streaming API be used with Apache Flume

[hussain_dream]

Hi all,

I am new to twitter APIs, I have used the search API with the Apache Flume but I am not sure if we can use streaming API with Flume.

So can anyone tell me if I can use the streaming API with Apache Flume?

[andypiper]

I haven’t personally used Flume, but from reading the documentation it appears that it does indeed work with the Streaming API. This looks like example code to listen to the sample endpoint.

[EricVinck]

I know 2 flume connectors for the twitter streaming endpoint (see below). There’s surely more available and/or you can code one yourself.

org.apache.flume.source.twitter.TwitterSource
this one is usually already part of your hadoop distrib

com.cloudera.flume.source.TwitterSource
this one must be downloaded from cloudera’s website

Hope this helps,
Eric.