Hello. I am new to Twitter API programming, and I am currently trying to collect a twitter social graph for early detection of trendy topics. The task is that, given 1M twitter user id’s, I need to collect the following relationships between them.
My approach is that I collect all followers of each of the 1M users. But then I figured that the current rate limit allows me only to request the follower ids 15 times per 15 minutes (once per minute). Although each request can list 5000 followers, I still need 1 request for a user even he/she might have only few followers, and even fewer belongs to the 1M users that I am interested. Under this constraint, I’ll be able to process at most 1 users per minute, or 694 days to process 1M users.
Is there any work around, like more IP’s, more accounts, some smart filtering, or anything that can let me complete the task in 2 months? Thank you in advance!