Streaming API and Rate Limits

[nambad]

Hi,

Can someone give more clarity on the rate limits the twitter streaming api impose? I am trying to build something where users can follow some userids and track some hashtags for sometime.

There will be very less number of users which would connect to a streaming server which in turn would open a stream on behalf of the user. Though the server would use different access tokens for different users accessing the stream with different tokens, its the same server connecting to twitter.

Will this end up in getting the account/accounts being rate limited? If yes, can someone kindly suggest another way to work this out?

Thanks,
Naman

[joncipriano]

Hi @nambad,

Your approach is correct. The limits you may run into are in regards to reconnecting to the stream and repeated attempts for users to login and attach to the stream with their tokens. You will want to make sure your application adapts to error responses accordingly and messages users appropriately. More information about techniques and error codes can be found in the link below.

Consuming streaming data

[davidlormor]

Hi @joncipriano,

I believe I’m trying to implement something similar to what the original poster asked, but wanted to make sure I’m understanding this correctly (after spending many hours scouring docs and this forum…).

So I’m building an app that will allow users to sign in via twitter and then connect to streams based on a particular hashtag. My app will open the streams based on each individual user’s credentials, which I would understand to be “user context.”

Does this mean that the 400 keyword rate limit would apply to each user of my application or to my application as a whole?

Thanks so much for any insight you can provide!

[joncipriano]

Hi @davidlormor,

The 400 keyword rate limit applies to your application as a whole. So, if 2 streams are opened on the behalf of 2 users tracking 200 keywords each, you would reach your rate limit for the application. Hope this helps!

[davidlormor]

Thanks for the quick answer @joncipriano! Any suggestions on how I can scale beyond the 400 keyword limit? All I could find was the link to certified partners, but don’t really see how that’s relevant…

[joncipriano]

Hey @davidlormor, you would be able to track more than 400 keywords by using services and products provided by GNIP on that certified partners page.

GNIP is a Twitter company and is how we provide enterprise level data streams to applications that need to process large amounts of Tweets at scale. For your purpose you would probably be interested in PowerTrack which supports thousands of filter rules. Keep in mind this is a paid product.

This level of data access puts us out of scope of this forum. If you would like to learn more, use the contact link on gnip.com. The API is different, but the documentation is excellent which can be found here.

[davidlormor]

Thanks again, @joncipriano! That looks like exactly along the lines of what I would need (and even appears to have resources for other social media sources I hope to expand to!) should the product take off. In the meantime I’ll continue hacking on the streaming API until I get enough users, but thanks for all the help.