Same problem here. Will add some more details for Twitter to troubleshoot on their side:
We have an app with two environments (one for staging and one for production). Yesterday night, data stopped coming in on the production environment, but continued to come in on the staging environment, through the ‘rules’ were exactly the same. To further investigate, I set a rule that should return a lot of data, and used to below script to connect to the streaming API:
import requests
import argparse
import time
import json
import datetime
def main():
aparser = argparse.ArgumentParser(
description='Twitter Filtered stream connection test'
)
aparser.add_argument(
'bearer_token',
help=(
'The bearer_token for the connection to twitter'
),
type=str
)
args = aparser.parse_args()
bearer_token = args.bearer_token
session = requests.Session()
session.headers.update({
"Authorization": "Bearer {}".format(bearer_token),
"User-Agent": "connectiontest_x26x55d"
})
# Get the rules, to print
response = session.get(
'https://api.twitter.com/2/tweets/search/stream/rules'
)
print('Using the following rules:')
print(response.text)
# Now start consuming. Just print id's to make sure it doesn't lag
params = {
'expansions': 'attachments.media_keys,author_id,geo.place_id',
'user.fields': 'id,location,name,username',
'media.fields': 'type,url,preview_image_url',
'place.fields': 'full_name,geo,id,place_type',
'tweet.fields': 'attachments,author_id,created_at,id,text,lang'
}
with session.get(
'https://api.twitter.com/2/tweets/search/stream',
stream=True, timeout=(1, 21),
params=params
) as response:
if response.status_code != 200:
raise requests.exceptions.HTTPError(
'Initial stream connection failed with code {}: {}'.format(
response.status_code, response.text
),
response=response
)
for response_line in response.iter_lines():
if response_line: # Also emtpy keep-alive signals are sent
payload = json.loads(response_line)
if 'data' in payload:
created_at = datetime.datetime.fromisoformat(
payload['data']['created_at'].rstrip('Z')
)
delay = (datetime.datetime.utcnow() - created_at).total_seconds()
print(
f"{int(time.time())}: "
f"Received id {payload['data']['id']}"
f" with delay {delay}"
)
else:
# e.g. errors
print(f"{int(time.time())}: {payload}")
else:
print(f"{int(time.time())}: Received empty line")
if __name__ == '__main__':
main()
I set the simple rule flood -is:retweet, which should return tweets on average every 5 seconds or so, but the output is the following:
python basic_test.py "REDACTED_BEARER_TOKEN"
Using the following rules:
{"data":[{"id":"1511678065394524165","value":"flood -is:retweet"}],"meta":{"sent":"2022-04-06T12:11:59.727Z","result_count":1}}
1649247140: Received empty line
1649247160: Received empty line
1649247180: Received empty line
1649247200: Received empty line
1649247220: Received empty line
1649247240: Received empty line
1649247260: Received empty line
As can be seen above, the script does have a connection, and does receive the keep-alive signals, but no actual data is being received. I used the bearer token of the actual production environment to test. The staging environment under the same project does not have issues.
@jessicagarson Can you perhaps forward the issues we’re seeing to the relevant support team?
Please note I experienced a similar issue a few months ago, in which case the staging environment was not getting data, but the production environment was getting it…
