I tried to collect all the tweets matching my requirement within one month. In the begining the code works well, while after 6 minutes, I got a "KeyError: ‘data’ ". Does anyone know why this happened? Many thanks.
It’s not possible to say without seeing in what context this code is running and on what data.
Maybe when reading a file there’s a blank line? or a different type of object if it was a stream?
Try printing the response.text or the reponse.json, it will show you the data that was returned. Also print the response.status_code. Note that the response here is the raw response you get, not the JSON you extract from it.
I am getting the same error, but my response shows that the result_count = 0, meaning no tweets were returned. The status code is 200 [OK]. I am wondering what happened here. The query definitely has more than zero tweets. I tried with different parameters also.
1 Like
I just realized that if you saved the output of a filter or sample stream, you are likely to gather control messages in the stream, which would not have a data key. The simplest way to deal with this is:
if "data" in response:
df = pd.DataFrame(response['data'])
... etc.
For turning twitter data into a Dataframe, i recommend twarc-csv - either as a command line tool, or as a library.
Many thank, I will try this.
I was thinking that the server might not return “data” because the frequency of requesting data is too higher? I wrote a loop to collect all the tweets satisfying my requirements, and I got the error after my codes runned 300 rounds in 404 seconds.
Thank you. Good to know that I am not the only one to have this problems. I think the solution from IgorBrigadir is good to try.
