I wanted to code up small personal project that would require use of the REST API, but it looks like my idea would hit the rate limits pretty quickly. I’ll explain one of the things I’d hoped to do that looks like it’d be problematic, and maybe you all can tell me if I’m missing something, or if there’s a good alternate approach. Here goes:
I want to count the number of times a User has been mentioned, by others, over the past year.
Looks to me like I’d need to use the API’s search to get mentions of the User, then cursor through the results til I get to my end date. Once I get to the end date I can just count all the results I got.
The search URL for this would, at its most basic and before encoding, be something like this:
This is straightforward, but if the User I’m interested in has thousands of mentions, I’d need to call the API tens of times. And if I want to do this on small set of Users (say, a few hundred), that’ll mean hundreds, probably thousands, of searches. And aside from this count of mentions, I’d have one or two other similar queries I’d want to run for each user. The point being, I’m way past the rate limit, with this approach.
So I guess what I’m asking (at great length–sorry) is…
Is there any way to run thousands of searches without hitting the rate limit? Is there any mechanism by which Twitter lifts rate limits for individuals?
Alternatively, am I missing some way to use the API to get metrics like this? Twitter Analytics looks like it has this sort of info… is there any plan to offer a public API for it?
ps. I realize could code around the limit by proactively idling my process when necessary, but that feels inelegant, and also it’d be a hassle, and anyway I suspect it’d fuzz up the accuracy of the results I’m looking for. Alternatively, I could register a handful of Twitter accounts for this purpose, and when one hits the limit, switch to the other… but that seems like an even bigger hassle and, to boot, like an abuse of the API.