Hello, I have the following curl statement:
curl --write-out '%{json}' -H "Content-type: application/json" "https://api.twitter.com/2/tweets/search/stream?expansions=author_id,entities.mentions.username&tweet.fields=author_id,entities,created_at,geo,id,in_reply_to_user_id,lang,public_metrics,possibly_sensitive,referenced_tweets,source,text&user.fields=created_at,description,id,location,name,pinned_tweet_id,protected,public_metrics,username,verified" -H "Authorization: Bearer $Bearen" -o historial.json
In short, I am saving in a json the tweets.
With a set of rules already added, so that it pulls me #covid19 tweets in Spanish.
The problem I’m having is the following:
I’m running the curl statement and I get the following error after a while. (usually after 5 hours or more)
As additional information:
- I am running this curl statement in a “git” command console from window 10.
- The time on my computer is correct and is synchronized.
- I have seen this error in the forum for API v1 and a possible cause could be several simultaneous connections (but this is not my case) I am only making one connection.
- In some errors I did get the reason inside the json in a more formal way. In this case it does not inform me of the error at the end of the .json.
- I’m new to this, it’s probably my fault
Thanks help!
This is a connection error of some sort, normally you would automatically reconnnect.
curl is not very suitable for this as it does not have any built in ways to deal with twitter specific issues like these.
I recommend using twarc instead, after you install it:
pip install twarc
twarc2 configure
The rules can be added like this:
twarc2 stream-rules add "#covid19 lang:es"
The stream can be opened with:
twarc2 stream output.json
