I’m an undergrad at Cornell and want to use Twitter posts by politicians to analyze their sentiment. I already have a list of user_ids but I don’t want to use my program to download them yet because I’m afraid I’ll hit the API limit and be banned. Is there a way around this? Perhaps allow one particular app to have a higher limit for academic purposes?
There won’t be any problems if you keep an eye on the rate limit headers returned after a request, and just wait for a few minutes whenever you run out.
How many user ids of politicians have you got? Hitting rate limits won’t get you banned. (but continuing to hammer away at the endpoint after you’ve received a Rate Limit notice is definitely not a good idea - existing libraries for R, Python etc all implement waiting for rate limits already)
If you’re getting tweets of specific users with https://dev.twitter.com/rest/reference/get/statuses/user_timeline definitely have a look at how you use App Only tokens. (You can make 300 requests to get/statuses/user_timeline with App Only Access)
You need just 16 requests to get all available tweets for a user: Max. 3200 retrievable tweets, with 200 tweets per request with count= parameter.
So you can fully process at least 18 politicians every 15 minutes (in practice this will be much faster, as many users have less than 3200 tweets)