#1

Hello,

I am looking into using a Premium API in my app to retrieve the live tweets published from a list of specific usernames. The size of my list of Twitter usernames includes about 50.000 accounts.

After some research into the API documentation I came across the Enterprise API PowerTrack ( PowerTrack API | Twitter API | Docs | Twitter Developer Platform), which seems to enable this.

I therefore contacted Twitter Enterprise and a representative gave me a quote which unfortunately is out of my budget. However, the Twitter representative suggested that there is a Premium API which allows to solve my use case.

I could not find a Premium version of the PowerTrack API.

I was considering using the Search Premium API but this would require too many API calls.
If I want new tweets every 30 minutes, I need to send many requests:
50.000(accounts) x 30(minutes) x 2(per hour) x 24(day) = 72.000.000 requests / day
Each request will give back the recent tweets from a specific Twitter account, not only the new ones, so these need to be filtered to extract only the new ones.

Do you have any suggestions how I can approach this?

Thank you in advance.

Regards,

Patrick

#2

Premium API would be a bad choice for this, if you want live tweets as opposed to historic data.

Since you’re someplace in the middle in terms of heavy / light usage - you may have to adopt an approach that uses several sources: Rest API, Streaming API, Search API:

First, i’d try to reduce or prioritise the size of the list of users - with 50,000 - most would be infrequently active, for these you can use Overview | Docs | Twitter Developer Platform to update them (this endpoint is limited to 100,000 per 24 hours). For the 5000 most active users, you can use the Filter stream POST statuses/filter | Docs | Twitter Developer Platform
You can also try to use the Standard Search API to search for from:user OR from:another for example Standard search API | Docs | Twitter Developer Platform

You’ll have to optimise, prioritise and merge these different sources together to get what you want.

#3

Thank you for you help but unfortunately I can’t reduce and prioritise the list of users as I can’t afford to lose any tweets.

I am considering changing my approach and, rather than getting live tweets, I would use the Premium Search API to get to recent tweets that match a specific keyword.
However, also for this scenario, I want to get back only the tweets posted by my list of accounts.

Reading the docs, I understand I can request tweets by the premium operators ‘keyword’ and ‘from’ but does this mean that I have to send a request for each account separately?
Is this is it then each time I would need to send 50.000 requests which is more than what is allowed by the API.

Thanks

#4

With the statuses/timelines approach you’re not really losing tweets - you’re just slowing down, where if you treat all 50,000 equally you can update them daily or so - more frequently if you check tweet counts with users/lookup or some other endpoint that gives you a user and check their tweet counts to only process users that tweeted.

For premium, you can define really long rules like “keyword (from:one OR from:two OR from:three)” to get keyword tweets from those specified users - the max length of a rule may be an issue, you you’ll still need to make separate calls, but not 50k.

#6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.