Regarding rate limiting and its effects

[pomacx]

We are currently developing a web application where we amongst other things present different people. We are concerned about how your rate limiting works in relation to our application and our users.

Our situation:
Currently a person is displayed in the web application, along with a one time request for a few of the persons twitter posts. We are expecting many users that display several people during an hour, thus many different twitter feeds will be fetched via unauthenticated client side ajax calls.

Our question:
Will your rate limiting only block the clients that “abuse” these calls, breaking your rate limit? Or will it with a growing amount of clients also eventually block our web application, being the referral, denying all our users from accessing your feeds from our page? In case the latter is true: is there a set limit or any pointers about how much traffic each user can generate before our service is blocked?

[episod]

I would really reconsider using authentication. It’s the proper way to scale an application like this, giving you the added benefit of server-to-server requests and the opportunity to use caching. You really will have no idea how many API calls your end-users have left when they arrive at your site and you’ll exclude a class of users on shared IP addresses almost completely.

[pomacx]

Lets say we use authentication (we have already considered doing this). There will still be quite a large amount of requests since it will be to alot of different feeds, and we want the feeds to be atleast somewhat updated so we can’t cache a feed for too long. How far can we go without getting blacklisted?

[episod]

If you’re using authentication and working on behalf of your users and staying within each token’s ratelimits, you can use the API as appropriate without blacklisting. Of course, it’s unwise to exhaust a token’s hourly limits in an extremely short period of time.

But it’s more likely that if there are a lot of “feeds” that you’re interested in, the REST API might not be the best source of data for you. Consider using the Streaming API instead. In particular, you may find the track follow methods interesting.

[niall]

[niall]