I have about 600k users, and I am interested in getting the most recent tweet for all these users. (Actually, I am interested in which users posted a tweet within 14 days but I don’t know if that makes the search any easier.)
What is the best way to do this?
If I make 600k calls to to get each user’s timeline, this is very inefficient because the queries are limited at 900/15m.
The best, most efficient way is to alter this to 7 days, because then you can use the 7day recent search with a bunch of queries of max length like from:user1 OR from:user2 OR from user3... up to 512 characters per query, and paginate over all such results. This will take a whole bunch of generating queries and splitting them but should be possible to do.
Alternatively, using v1.1 API, you can use the user lookup API to query each user and get the latest tweet embedded in the user object: GET users/lookup | Docs | Twitter Developer Platform you can look up 100 user IDs per call.
