Going a bit insane here…
I’m modifying my bot @songfinder_bot to use the V2 endpoints finally, but in my testing, despite what I’ve tried, the stream is not returning the correct tweets.
Checking my filtered stream rules only shows I have one rule, as it should be:
$ curl -X GET 'https://api.twitter.com/2/tweets/search/stream/rules' -H "Content-type: application/json" -H "Authorization: Bearer $BEARER_TOKEN"
{"data":[{"id":"1546763511157510145","value":"@songfinder_bot"}],"meta":{"sent":"2022-07-12T08:02:34.275Z","result_count":1}}
However when I initiate a stream with the EXACT same Bearer token, right after checking that my rules are correct, it yields irrelevant tweets, e.g:
{'id': '1546768745334439936', 'text': 'RT @AudiOfficial: Like or share: Express yourself with the Audi TT RS Coupé. Like for Python Yellow. Share for Dew Silver. https://t.co/zDV…'}
{'id': '1546768746479374336', 'text': 'RT @CrownedHead06: Things you can become after learning Python 🐍 ... #ML #DataScience #SQL #Cybersecurity #BigData #Analytics #AI #IIoT #Py…'}
Looking at what all the returned results have in common, it seems to be filtering results for python. NOWHERE in my code have I chosen to return that query. The application I’m using is just one I use for testing things, so maybe at some point in time I DID set it to return all tweets with python in them, however it shouldn’t be doing that anymore, as I’ve verified what rules I want to filter for already.
I’ve double-triple-quadruple-checked that I’m using the correct Bearer token, so I’m going a little crazy trying to work out what I’m doing wrong.
So then I stripped down all the code, and I tested the stream using this script, to make 100% sure I’m not doing something wrong. It deletes any existing rules and then sets the EXACT rule I want.
Here is the exact program I ran:
import requests, json
BEARER_TOKEN = ""
headers = {"Authorization": f"Bearer {BEARER_TOKEN}"}
response = requests.get("https://api.twitter.com/2/tweets/search/stream/rules", headers=headers)
current_rules = response.json()
print(current_rules)
if current_rules is None or "data" not in current_rules:
pass
else:
ids = list(map(lambda rule: rule["id"], current_rules["data"]))
payload = {"delete": {"ids": ids}}
response = requests.post("https://api.twitter.com/2/tweets/search/stream/rules", headers=headers, json=payload)
print("Deleted existing rules")
sample_rules = [{"value": "@songfinder_bot"}]
payload = {"add": sample_rules}
response = requests.post("https://api.twitter.com/2/tweets/search/stream/rules", headers=headers, json=payload)
response = requests.get("https://api.twitter.com/2/tweets/search/stream", headers=headers, stream=True)
print(response.status_code)
for response_line in response.iter_lines():
if response_line:
json_response = json.loads(response_line)
print(json_response['data'])
But still, I get stupid results like this:
{'id': '1546773459522445315', 'text': 'RT @KirkDBorne: [Download FREE 1213-page PDF eBook] "Learning Python" (4th Edition) at https://t.co/u9KcUcpX50\n————\n#BigData #DataScience #…'}
{'id': '1546773459509952512', 'text': 'RT @JobPreference: NEED a #JOB?\nSign up now https://t.co/o7lVlsCHXv\nFREE. NO MIDDLEMEN\n#Robotics #AIEthics #MachineLearning #AI #Python #Da…'}
What is happening. Have I found some sort of bug? I have no idea if the issue is reproducible with other apps or not. The results I’m getting make absolutely no sense. Have I really done something wrong, and I’m unable to see it? I’m seriously lost… Streaming on v1.1 has no issues for me.
Thanks in advance for any help.
