I’m trying to do a full archive search, hoping to get ~1000s of tweets going back some months. My limit per call is set to 100 but I only get back about 30 tweets and the latest is about 7 days back. The metadata returned from collect_results() also doesn’t have any information for paging e.g. earliest tweet ID.
When I try to specify the date range (start_time in gen_request_parameters) I get an error that start_time cannot be more than 7 days back. It’s as if I am doing a ‘recent’ search limited to a 7 day window and not the full archive search.
Anyone know how I can activate the full time range search?
query = gen_request_parameters(q, results_per_call=100,tweet_fields=‘text,author_id,id,created_at’,
expansions=‘author_id’,start_time=‘2020-01-01’)
results = collect_results(query,
max_tweets=500,
result_stream_args=search_args
)
Looks like you are using https://github.com/twitterdev/search-tweets-python/
Make sure you are using the v2 version, https://github.com/twitterdev/search-tweets-python/tree/v2 which is installed from searchtweets-v2 · PyPI
run
pip uninstall searchtweets
pip install --upgrade searchtweets-v2
to make sure.
Full archive search endpoint should be specified in your yaml config too, see here for a full example:
(there will be a new version of search-tweets-python for v2 that will be able to deal with expansions slightly better, it’s not released yet, but the code is in the branch)
Great! Yes the issue was with the YAML file, I was using the ‘recent’ endpoint instead of the ‘all’ endpoint.
Many thanks!
1 Like
