Trying to retrieve retweets from the 30 day premium archive using searchtweets is resulting in:
HTTP Error code: 422: Unprocessable Entity: This is returned due to invalid parameters in a query or when a query is too complex for us to process. –e.g. invalid PowerTrack rules or too many phrase operators, rendering a query too complex.
This is my workflow:
- Installed the python3 version of searchtweets using pip
- Cloned the repo and changed directory to the tools folder.
- Created a ~/.twitter_keys.yaml:
- Run the command
python3 search_tweets.py --filter-rule "972890661115457537 is:retweet" --print-stream --debug
Where 972890661115457537 is the tweet id of a tweet by Bernie Sanders. It was made on 11th MArch 2018.
This is the output
DEBUG:root:{
"results_per_file": 0,
"config_filename": null,
"max_results": 500,
"account_type": null,
"from_date": null,
"credential_yaml_key": null,
"pt_rule": "972890661115457537 is:retweet",
"credential_file": null,
"debug": true,
"env_overwrite": true,
"max_pages": null,
"to_date": null,
"results_per_call": 100,
"count_bucket": null,
"filename_prefix": null,
"print_stream": true
}
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.twitter.com
DEBUG:urllib3.connectionpool:https://api.twitter.com:443 "POST /oauth2/token HTTP/1.1" 200 152
WARNING:searchtweets.credentials:Grabbing bearer token from OAUTH
DEBUG:root:{
"pt_rule": "972890661115457537 is:retweet",
"results_per_file": 0,
"bearer_token": "<Token>",
"env_overwrite": true,
"max_results": 500,
"results_per_call": 100,
"debug": true,
"print_stream": true,
"endpoint": "https://api.twitter.com/1.1/tweets/search/30day/TwitteRT.json"
}
DEBUG:root:{
"pt_rule": "972890661115457537 is:retweet",
"results_per_file": 0,
"bearer_token": "<Token>",
"env_overwrite": true,
"max_results": 500,
"results_per_call": 100,
"debug": true,
"print_stream": true,
"endpoint": "https://api.twitter.com/1.1/tweets/search/30day/TwitteRT.json"
}
DEBUG:root:ResultStream:
{
"rule_payload":{
"maxResults":100,
"query":"972890661115457537 is:retweet"
},
"tweetify":false,
"username":null,
"endpoint":"https:\/\/api.twitter.com\/1.1\/tweets\/search\/30day\/TwitteRT.json",
"max_results":500
}
INFO:searchtweets.result_stream:using bearer token for authentication
DEBUG:searchtweets.result_stream:sending request
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.twitter.com
DEBUG:urllib3.connectionpool:https://api.twitter.com:443 "POST /1.1/tweets/search/30day/TwitteRT.json HTTP/1.1" 422 318
WARNING:searchtweets.result_stream:retrying request; current status code: 422
DEBUG:searchtweets.result_stream:sending request
DEBUG:urllib3.connectionpool:https://api.twitter.com:443 "POST /1.1/tweets/search/30day/TwitteRT.json HTTP/1.1" 422 317
WARNING:searchtweets.result_stream:retrying request; current status code: 422
DEBUG:searchtweets.result_stream:sending request
DEBUG:urllib3.connectionpool:https://api.twitter.com:443 "POST /1.1/tweets/search/30day/TwitteRT.json HTTP/1.1" 422 318
WARNING:searchtweets.result_stream:retrying request; current status code: 422
DEBUG:searchtweets.result_stream:sending request
DEBUG:urllib3.connectionpool:https://api.twitter.com:443 "POST /1.1/tweets/search/30day/TwitteRT.json HTTP/1.1" 422 317
ERROR:searchtweets.result_stream:HTTP Error code: 422: Unprocessable Entity: This is returned due to invalid parameters in a query or when a query is too complex for us to process. –e.g. invalid PowerTrack rules or too many phrase operators, rendering a query too complex.
ERROR:searchtweets.result_stream:rule payload: {'maxResults': 100, 'query': '972890661115457537 is:retweet'}
Traceback (most recent call last):
File "search_tweets.py", line 190, in <module>
main()
File "search_tweets.py", line 184, in main
for tweet in stream:
File "/usr/local/lib/python3.5/dist-packages/searchtweets/result_stream.py", line 202, in stream
self.execute_request()
File "/usr/local/lib/python3.5/dist-packages/searchtweets/result_stream.py", line 253, in execute_request
rule_payload=self.rule_payload)
File "/usr/local/lib/python3.5/dist-packages/searchtweets/result_stream.py", line 101, in retried_func
raise requests.exceptions.HTTPError
requests.exceptions.HTTPError
Also is it possible to get all the names of the retweeters of a tweet? This is required to create a network of how the information is dissipated.