I’ve upgraded to v1.1 of the API, but now I have the issue of hitting the rate limit before even getting all the friends for a larger user. I used to get around some of the rate limit issues with a whitelisted IP and app, and I’d use that to get the friend list, but that’s no longer the case with v1.1. Let’s say I want to retrieve the friends list of a person with 250,000+ friends (same goes for followers - imagine 1 million+!) - at 5,000 per page, that’s at least 50 API calls to get the entire friends list! At 15 calls per 15 minutes for that API call, getting one of these users’ list of friends becomes impossible!
What is Twitter’s recommendation for this? Is there a new way of getting whitelisted? Are we no longer allowed to get the friend lists of larger users? Do I have to only call 75k at a time and just cache the results (which is really quite a pain, especially since we can’t guarantee the newest ids will always be the newest followers according to the docs). What do you suggest?