Twitter premium API provides only 100 tweets per request even though limit should be 500

[lusearch1]

Dear Community,

Yesterday, I upgraded my developer account to premium access for the fullarchive search. While I was using the API in sandbox for testing with a tweet limit of 100 per request, I thought the upgrade would change that.
However, I just downloaded 5000 tweets for a query for a single day in 2020 - just to discover that the Twitter API has divided these 5000 tweets into 100 per request. Which means I now have lost 50 requests via the API - even though it should only be 10 requests according to the Premium API conditions.

What is the issue here? I am using the rtweet package to access the Twitter API and had set my limit per query to 500 for 10 iterations.

Thank you in advance!

[andypiper]

It may be the rtweet library itself - is that capable of asking for 500 tweets?

[IgorBrigadir]

If you used rtweet, there is a long standing issue with this maxResults parameter in rtweets set to 100 default · Issue #383 · ropensci/rtweet · GitHub it’s hard coded to 100 results (sandbox limit) instead of 500 but there’s a workaround / fork in the comments in github.

Or you can try using the python code GitHub - twitterdev/search-tweets-python: Python client for the Twitter 'search Tweets' and 'count Tweets' endpoints (v2/Labs/premium/enterprise). Now supports Twitter API v2 /recent and /all search endpoints. that has a command line tool,

Or you can try the python code above via R but this can take a bit of setting up because of different environments: Running the python package for search tweets in r | Twitter Developer Platform