Why does the search rate limit reset time sometimes go BACKWARDS?


#1

My app carefully gets its current rate limits via application/rate_limit_status, and then carefully keeps track of how many requests remain for /search/tweets as it pages through results. Several times a day, however, it gets a “Rate limit exceeded” error, frequently on its very first search request. I’ve started logging the results of rate limit lookups immediately before and after I get the unwarranted error, and there are crazy disparities. Most of the time, rate_limit_status suddenly says the search limit is now zero, when seconds ago it said it was 180. But occasionally there’s additional craziness, where the reset time is suddenly EARLIER than it was a short time ago:

2013-12-30 02:49:01,053  search  20  Rate limit /search/tweets: {u'reset': 1388390641, u'limit': 180, u'remaining': 180} (reset in 15.0m)
...
2013-12-30 02:49:32,713  search  30  Rate limit now {u'reset': 1388389802, u'limit': 180, u'remaining': 0} (reset in 0.5m)

I’ve elided some other logging, but in between these two entries the app makes a single request to the search API, which gets a “Rate limit exceeded” error.

What could explain this? And what is best practice for dealing with the apparent unreliability of the rate limit API?

Thanks.