Since and Until date option in Tweepy

nshashank_02 · 2022-11-29T05:41:57.062Z

Hello, I am working on Tweepy and I need to collect tweets for just specific dates, not the recent ones as Tweepy does. I tried using the “start_time” and “end_time” features but it showed as “Unexpected parameter: start_time”.The “start_time” and “end_time” features work in Postman API but not in Python code. So, can anyone help me with the problem!!

kato_shinya · 2022-11-29T06:12:00.922Z

Hi @nshashank_02 ,

What kind of endpoint are you using?

Or, what kind of function are you using in tweepy?

nshashank_02 · 2022-11-29T09:05:51.506Z

Well, I am getting some results. I did use the endpoint “https://api.twitter.com/2/tweets/search/all” and I’m getting the results. But, without using any endpoint is it possible? Because I’m getting tweets without the start date and end date input using Tweepy. But, if I want to have tweets from a specific date, I want to use them. So, any suggestions regarding that? I’ll share the screenshot of the code without using the endpoint.

Screenshot (623)1366×768 121 KB

Screenshot (624)1366×768 116 KB

Screenshot (625)1366×768 122 KB

Screenshot (626)1366×768 127 KB

Screenshot (627)1366×768 124 KB

kato_shinya · 2022-11-29T09:22:09.341Z

I have never used tweepy, but looking at the latest tweepy implementation, it seems that the following methods support the /2/tweets/search/all endpoint and the /2/tweets/search/recent endpoint.

github.com

tweepy/tweepy/blob/e2fb467b52659e87de592ff80b03edb9324c0189/tweepy/client.py#L1070


      
              id = self._get_authenticating_user_id(oauth_1=user_auth)
              route = f"/2/users/{id}/retweets"
          
          
    return self._make_request(
                  "POST", route, json={"tweet_id": str(tweet_id)},
                  user_auth=user_auth
              )
          
          
# Search Tweets
          
          
def search_all_tweets(self, query, **params):
              """search_all_tweets( \
                  query, *, end_time=None, expansions=None, max_results=None, \
                  media_fields=None, next_token=None, place_fields=None, \
                  poll_fields=None, since_id=None, sort_order=None, \
                  start_time=None, tweet_fields=None, until_id=None, \
                  user_fields=None \
              )
          
          
    This endpoint is only available to those users who have been approved
              for the `Academic Research product track`_.

https://github.com/tweepy/tweepy/blob/e2fb467b52659e87de592ff80b03edb9324c0189/tweepy/client.py#L1173

And I guess search_tweets you’re using is for v1.1 endpoint.

github.com

tweepy/tweepy/blob/92f9cb94865c0c991428772d2f36efb795308ab2/tweepy/api.py#L1217


      
                      'POST', 'statuses/update_with_media', endpoint_parameters=(
                          'status', 'possibly_sensitive', 'in_reply_to_status_id',
                          'lat', 'long', 'place_id', 'display_coordinates'
                      ), status=status, files=files, **kwargs
                  )
          
          
# Search Tweets
          
          
@pagination(mode='id')
          @payload('search_results')
          def search_tweets(self, q, **kwargs):
              """search_tweets(q, *, geocode, lang, locale, result_type, count, \
                               until, since_id, max_id, include_entities)
          
          
    Returns a collection of relevant Tweets matching a specified query.
          
          
    Please note that Twitter's search service and, by extension, the Search
              API is not meant to be an exhaustive source of Tweets. Not all Tweets
              will be indexed or made available via the search interface.
          
          
    .. note::

So is it possible to use search_all_tweetsl or search_recent_tweets instead? Then you can use start_time and end_time

IgorBrigadir · 2022-11-29T15:01:41.406Z

The main issues are: You need Academic Access to search for tweets with a specific date that’s not the last 7 days. If you did not apply for Academic Access separately to your developer account and do not have a separate Project on your dashboard, you do not have academic access Twitter Developer Access - twarc

Also you are using v1.1 APIs, not v2. Academic access is for v2 only. Tweepy does support this, but using tweepy.Client Client — tweepy 4.12.1 documentation only - tweepy.API API — tweepy 4.12.1 documentation is for v1.1 Only.

Your code also expects v1.1 Format tweets, which may not work in v2.

Twarc can be used here too, maybe with fewer code changes than for tweepy if you want to try that too: Examples of using twarc2 as a library - twarc