#1

Hi all,
I am trying to download all Tweets (from a single user) posted to a public account through R.
I’ve been using the rtweet package on R, and used the get_timeline function to get the most recent 3,200 Tweets posted to this account.

My issue is there are just over 4000 Tweets posted to this account, and I’d ideally like to access them all for my research. I understand the get_timeline function has a 3200 Tweet restriction, but I’m wondering if there is another way to access those remaining ~ 900 Tweets? Would I need to buy a Premium API or is there an easier way to do this that I’m missing?

I’ve also tried doing this via Postman but I kept getting error codes when I tried running an archive search of the account, so I gave up and went back to R.

I’d appreciate any help - I’m very new to Twitter API and it’s been quite a learning curve :slight_smile:

1 Like
#2

If you have Academic Access you should be using the v2 Search, which rtweet does not support unfortunately. But otherwise a Premium call to fullarchive with rtweet will work.

I would recommend using twarc for downloading tweets in the command line and importing them into R if you’re using Academic Access

2 Likes
#3

Hello Igor, I installed twarc and am trying to configure it. For the academic access, I was given by my supervisor the (1)appname (2)key (3)secret (5)bearer token. It appears that the access token and access secret are missing, but instead there is a ‘bearer token’. Does this look familiar to you? I am running into an error when trying to configure twarc. Thank you for your help!

1 Like
#4

How exactly are you configuring twarc and what errors are you seeing? (twarc configure is a different command for v1.1 API only.)

what happens when you run

twarc2 configure

And enter your bearer token? (paste it in and press enter, the input gets hidden). The bearer token is the only thing you need, because the API key and API Secret is used to generate a Bearer token. You can also enter these if you wish.

#5

Thank you for your quick reply, and for the heads up that the input gets hidden when I paste it, I thought my keyboard wasn’t working at first.

When I enter the bearer token, it prompts (Optional) Add API keys and secrets for user mode authentication [y or n]. I first tried “y” and entered the key and secret that were given to me, but then it printed Error: Unable to authenticate. I then tried the process again, but answering “n” and it printed Your keys have been written to C:\Users… but when I look in that directory, no such folder exists. I am also concerned that the key and secret may have changed now, and the original ones I was given might now be invalid, is this true?

On a side note, whenever I wand to run “twarc2 configure”, I first have to change the directory to a really specific folder where the twarc2 file is located (C:\Users\David\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\Script). I would rather just copy one of those folders, maybe the Script folder to C:\Windows\System32 so that I can use it from the default directory of the command prompt. Would you see any issues with that?

Thank you for your help!

#6

For the specific folder thing, have a look here: Windows 10 - twarc (The part about PATH)

this config is a file with no extension, so it may appear in the folder just above. It should contain something like this:

bearer_token = 'AAAAAAA....zzzzzzzz'

(with your entered bearer token)

To make sure it’s working, you can try a test call:

twarc2 search --archive --limit 100 "dog" dog_test.json