How to stream tweets using tweepy from a start date to end date using python?

python

#1

I am currently in the process of doing some research using sentiment analysis on twitter data regarding a certain topic (isn’t necessarily important to this question) using python, of which I am a beginner at. I understand the twitter streaming API limits users to access only to the previous 7 days unless you apply for a full enterprise search which opens up the whole archive. I had recently been given access to the full archive for this research project from twitter but I am unable to specify a start and end date to the tweets I would like to stream into a csv file. This is my code:

import pandas as pd
from tweepy import Stream
from tweepy import OAuthHandler
from tweepy.streaming import StreamListener

ckey = 'xxxxxxxxxxxxxxxxxxxxxxx’
csecret = 'xxxxxxxxxxxxxxxxxxxxxxx’
atoken = 'xxxxxxxxxx-xxxxxxxxxxxxxxxxxxxxxxx’
asecret = ‘xxxxxxxxxxxxxxxxxxxxxxxxxxxx’

=============================================================================

def sentimentAnalysis(text):

output = ‘0’

return output

=============================================================================

class listener(StreamListener):
def on_data(self, data):
tweet = data.split(’,“text”:"’)[1].split(’","source’)[0]
saveMe = tweet+’::’+’\n’
output = open(‘output.csv’,‘a’)
output.write(saveMe)
output.close()
return True

def on_error(self, status):
    print(status)

auth = OAuthHandler(ckey, csecret)
auth.set_access_token(atoken, asecret)
twitterStream = Stream(auth, listener())
twitterStream.filter(track=["#weather"], languages = [“en”])

Now this code streams twitter date from the past 7 days perfectly. I tried changing the bottom line to

twitterStream.filter(track=["#weather"], languages = [“en”], since = [“2016-06-01”])

but this returns this error :: filter() got an unexpected keyword argument ‘since’.

What would be the correct way to filter by a given date frame?


#2

The streaming API is always real-time. You cannot search backwards, you can only start to listen for results from the moment your app connects so I am not sure why you have this impression that you only have a 7 day look back window. There is no since filter on the real-time streaming api, that’s only a search option, and only for 7 days on the free search API.


#3

I have full archive access, won’t I be able to choose a start and end date (as the premium search API lists these kinds of parameters in the documentation) with fromDate and toDate? Unless I am sorely mistaken on how I can use this full archive search


#4

Tweepy’s streaming support is for the standard streaming API, not premium full archive. Try a library like search-tweets. https://github.com/twitterdev/search-tweets-python