#1

Hi Twitter developers,

I’m currently trying to use the rtweet package through R to pull tweets regarding certain stocks from the past (using an academic API account).

I’m getting the following error:

Screenshot 2021-03-12 at 12.16.24
Screenshot 2021-03-12 at 12.16.241762×244 63 KB

I was under the impression that an academic account would allow me to extract more tweets. I have no way of contacting Twitter directly, so hopefully someone here could help to clarify?

Many thanks

#2

search_fullarchive

rtweet does not support Academic Access v2 endpoint, sounds like it’s the same issue and solution as here:

2 Likes
#3

Thanks for your reply!

So, by adapting the code in that example, I should be able to bypass this issue?

#4

Yes, the script in Get tweets from the Academic Research product track. · GitHub should work - rtweet will not work.

2 Likes
#5

Thanks, you’ve been a huge help.

1 Like
#6

Hello!

Does any of you have an example of the application of the Github code?

I have been trying to implement it myself but I can’t figure out what is wrong.

Thank you

#7

@LuisigMenendez if that snippet isn’t working for you, someone made a separate R package for this: GitHub - cjbarrie/academictwitteR: Repo for academictwitteR package to query the Twitter Academic Research Product Track v2 API endpoint.

also see this reply:

1 Like
#8

Hi there,
I’m facing the same challenges of extracting the full archive of historical (more than 9 days old) tweets. I tried two R packages: rtweet and academictwitteR. And here’s my experience:

  1. rtweet: use search_fullarchive always works, but the major problem is, every time only 100 tweets can be downloaded. For academic researchers who usually work with millions of tweets, downloading 100 tweets per batch isn’t a practical way.
  2. academictweeteR: use the command get_all_tweets. This command will return a nested data frame that requires a lot of work to release the elements from the nested lists. It’s actually Okay to work with nested data frame as long as the command functions well. However, my experience is that for 95% of the time, I received the error code 503 and couldn’t download anything. After asking the package developers, they said that it’s the problem of Twitter rather than their package.
#9

This does not use the correct API. That function calls the v1.1 Premium API Premium search APIs | Docs | Twitter Developer Platform not the v2 Academic access API GET /2/tweets/search/all | Docs | Twitter Developer Platform

Yes, this is an error on the API side, but the workaround I found that fixes this is to specify a smaller request size with max_results set to 100 or 250 instead of 500

However, my recommendation is to not use R for retrieving the tweets, and instead use twarc2 command line tool and import the data into R for analysis: