Twitter strategy: Streaming API vs. REST API


#1

I’m working on a kind of a twitter wall. Users can login with twitter and create their own wall, which will display the tweets for certain terms/hashtags.

I’m still looking for the best strategy to get the data out of the Twitter APIs.
Following some of my thoughts:

Strategy 1: Streaming API

  • Open a single stream (POST statuses/filter) for all walls
  • Each hashtag is added to the track parameter
  • When new tweets arrive, they will be processed and sent to the corresponding wall
  • (“one account, one application, one open connection” cf. https://dev.twitter.com/discussions/14935)

Problems with the Streaming API

  • Streaming api is limited to 400 keywords to track

  • What to do if there are more than 400 keywords to track?

  • Streaming api is limited to 1% of the tweets of the firehose

  • It’s very difficult to get above 1% of the firehose, but if you’re tracking a term like “apple” it’d be pretty easy to exceed the 1%. (cf. https://dev.twitter.com/discussions/6349)

  • How can I handle such popular terms? Blacklist them?

Strategy 2: REST Search API**

Problems with the REST Search API

  • Polling
  • Could get very expensive to poll the API for a lot of users.

Do you have any suggestions/recommendations which strategy would fit the best? Are there already solutions for these problems?

Best regards


#2