#1

Hello,

I ran search_all_tweets using Tweepy to grab all historical tweets for a set of users for a specific time period. I ran the exact same code twice in November 2021 and December 2021. When I compared the returned tweets from two datasets, I found out that for some users, there are differences in the returned tweets for the same time period.

One possibility for the differences may be from users deleting certain tweets between November 2021 and December 2021. However, when I identified the different tweets between two datasets, some of the tweets are from a year or two years ago and it seems odd that a user would delete a tweet that they posted such a long time ago between November 2021 and December of 2021.

Also, for some users, I actually got more tweets in the December 2021 run. When I identified these increased tweets, these tweets are also from a long time ago.

My question is as follows. Did anyone experience the same issue? If so, is it possible that Twitter API V2 may randomly drop certain tweets?

Many Thanks in advance!

My sample code is as follows:

for name in list:
       for response in tweepy.Paginator(client.search_all_tweets, 
                                 query = 'from:'+ name +' lang:en', 
                                 user_fields = ['username', 'public_metrics', 'description', 'location'],
                                 tweet_fields = ['created_at', 'geo', 'public_metrics', 'text', 'attachments','referenced_tweets', 
                                                 'entities'],
                                 media_fields = ['type', 'preview_image_url', 'public_metrics'],
                                 expansions = ['author_id','referenced_tweets.id','attachments.media_keys'],
                                 start_time = '2006-03-21T00:00:00Z', # YYYY-MM-DD-HH:MM:SS
                                 end_time = '2021-11-05T00:00:00Z',
                                 max_results=500):

        time.sleep(2) 
        result_tweets.append(response)
1 Like