POST statuses / filter limits

[blueparrots]

I have subscribed to the public stream and am tracking one keyword.

I know that if this keyword exceeds 1% of the firehose stream I will get a limit error message.

The question: Does this filter filter against the firehose stream, or against a 1% sample of the firehose stream?

I notice when I save all status/sample messages and search various fields for my keyword I see substantially fewer matches than when I use the status/filter pattern.

[romainhuet]

Hi @blueparrots,

Thanks for the question. When you use the statuses/filter endpoint, you are filtering from the full firehose, with a maximum resulting volume of 1% of the total Tweets at that moment.

In other words, if the keywords you are tracking account for less than 1% of the firehose, you will receive all the matching Tweets, otherwise you will be capped around 1%.

There are more than 500 million Tweets posted every single day on Twitter, so 1% of those is already a very large number!

Best,
Romain

[blueparrots]

Awesome thanks!

[lucascolin]

Hi @romainhuet,

If our stream (statuses/filter/follow) hits the 1% rate limit will we get a notice every time it does? How will we know if we are hitting the limit?

thanks,
Luke

[blueparrots]

@lucascolin

If you hit the 1% rate limit you’ll receive a response like this:

{“limit”:{“track”:1234}}

Where track is the number of missed messages.

To test this, try subscribing to the status/filter stream while tracking a common word such as “the.” You’ll hit the limit pretty quickly.