Millions of request instead of streaming, is it the new way?

[savageman86]

Let’s say I need to retrieve mentions timeline 1000 Twitter accounts. Previously I would receive around 2000 mentions per day for those 1000 accounts.

I’m not paying $10.000 per month for the new Premium API to retrieve 2000 mentions per day. So I need to use the Standard REST API. Polling at rate limit would need me to generate 1 request every 12 seconds for each account (75 reqs/15 min).
86400/12 = 7200 GET requests per day per account, or 7.2 millions GET requests for 1000 accounts.

With at most 2000 requests returning a useful result (because I have 2000 mentions per day on average), that’s a “hit” rate <0.03% leaving me with more than 99.97% of those requests done only to waste CPU and bandwidth both for me and for Twitter backend.

I don’t see how it’s worth even for Twitter, is that how we’re supposed to do now?

[andypiper]

Hi! It’s not completely clear from your question how you intend to work with the Twitter developer APIs, or your use case. Would you be able tell us more about that?

Thank you

[savageman86]

I used to receive mentions in real time using user streams and I am evaluating other available API to do the same.
Monitoring ~1000 accounts means maintaining 1000 HTTP long-lived user stream connections (great!). I didn’t find any better solution than to use the polling API and send 7 millions request per day to be as close as possible as my current user stream implementation.

[andypiper]

The standard API is not really built to support that use case, but you’re welcome to use it inside of the available rate limits. For reliable realtime delivery of mentions, we would suggest that the Account Activity API is the best alternative. For the scale and number of users you’re describing you could contact us about the enterprise plans. Another alternative might be to use the search API to find user mentions.

[savageman86]

AAAPI is too expensive, I can’t use it. I submitted the enterprise form twice already, one several months ago with my personal account (which was wrong, but I can’t find how to “cancel” it). And another time with my company account a few weeks ago after we registered the developer account including Beta Access to AAAPI (definitely more than a month ago) but never got a reply.

We have to use what’s available now that the API are regressing. I definitely don’t need reliable real-time and ~15s delay is much acceptable. It’s worse than the old user streaming API and >99.97% not efficient but I don’t see any other solution.