I’ll add IDs in a bit, but here’s what we’re doing:
We provide detailed follower-base analytics (VIPs following you, most influential by followers, etc.). User authenticates, we scan a his follower base with (rate-limit auto-scaling) requests for “followers/ids”. If exhausted, we back off and wait and return later. This endpoint is wonderful, supplies us with 5.000 IDs a pop, we can easily scan through very large accounts (our users with 1M+ followers). Once we have that (i.e. a full list of everyone following this person), we need details to run analytics. So we scan through the large list of IDs and use “users/lookup” with 100 IDs supplied on each call, also rate-limit adjusted. If an error occurs, we reduce to 50 IDs, if still fails, we reduce to 10 IDs, if still fails, we reduce to 1 ID and if that fails we’ll remove this ID forever (poisoned) and scale back up to 100 IDs requested on next call.
The issue is here:
We noticed that the request for 100 IDs (on larger accounts, somewhere towards the middle/end of bucket) via users/lookup responds with only 1 ID, instead of 100. Ok, that’s fine, maybe 99 invalid since last followers/ids request an hour ago? Maybe a reason to back off and request only 1 next call? Not sure. If 99 were dead accounts and therefore not returned, that would be fine (although why return them with followers/ids right before, then…). But the strange thing is that when I keep bugging the API with more 1 ID calls (instead of 10,50 or 100 IDs max), it will supply them, just only 1, but will go further forever almost it seems until it hits rate limit (which is fast, after 350 calls basically in that case).
(1) [followers/ids] Get all follower IDs for a user, 5000/call, take care of rate limits. Loop until done.
(2) DONE with all IDs, temporarily saved on our side. Now process these for details.
(3) [users/lookup] Request details for 100 IDs [example IDs: 5000,5001,5002,5003…5099]
(4) API responds with 1 ID instead of 100. Ok, save details for that one.
(5) [users/lookup] Request details for 100 IDs [now: 5001,5002,5003…5100]
(6) API responds with 1 ID instead of 100. Ok, save details for one.
(7) [users/lookup] Request details for 100 IDs [now:5002,5003,5004…5101]
(8) API responds with 1 ID instead of 100. Ok, save details for one.
So it seems 5001,5002,5003 were actually valid. I still only got 1 ID (ID=5000) on the first call. So I figured, as said earlier, it might be a smart way for you guys to deal with traffic / response times (if time out, just return 1 ID instead of none) – but even if I dial down the requested IDs to 50, or 10, it will just return 1 ID for these larger accounts, towards the end.
So, if you have an (authenticated) account like @sacca - it’s basically impossible to run users/lookup and get a complete picture of his most popular followers.
I’ll compile a list of IDs and a better explanation, but this is what we’re seeing right now. Thanks again, your support here on this forum is fantastic.