What is the best practice to get missing tweets using Streaming API

chirgeo · 2013-05-28T10:29:09.000Z

Hi!

I’m trying to implement a simple script which get tweets from user timeline using Streaming API.

Now, as long as the script is up and running, connection is opened, I get tweets without problems.

Now, imagine that we have a server restart or any other reasons which make the script not running for some period.

What is the best practice to get those missing tweets?
Should I use REST API in combination with Stream API or only REST API?

Thx.

episod · 2013-05-28T15:01:26.000Z

Yes, most leverage both APIs or a third party provider of data. Take note of the most recent tweet IDs you consumed prior to disconnecting. Note the tweet IDs of the tweets you receive once reconnecting. Use these as lower and upper bounds of since_ids and max_ids when leveraging search/tweets or statuses/user_timeline or other methods. Keep in mind that the search API is not exhaustive… there will be some tweets you just won’t be able to easily find if you miss them.

chirgeo · 2013-05-29T06:54:56.000Z

Thx for info. I’m also planning to use the same methods you described.

What exactly you mean by “there will be some tweets you just won’t be able to easily find” ?

episod · 2013-05-29T14:47:59.000Z

The Search API has a limited index of tweets – it doesn’t index all of them and the way it responds to queries is assuming there’s a human on the other side wanting tweets to engage with based on the query provided – it’s not meant to be a pure source of data.