Can't get simple search working with Premium API

[miguelepdsilva]

I’ve just got access to the Premium API (Sandbox) and I’m learning how to use it. I followed the steps in https://twitterdev.github.io/search-tweets-python/index.html but I’m getting the following error: “HTTP Error code: 422: Unprocessable Entity: This is returned due to invalid parameters in a query or when a query is too complex for us to process. –e.g. invalid PowerTrack rules or too many phrase operators, rendering a query too complex.”

The query I’m testing is simply: “#vaccines”. I’m testing the full archive sandbox (I labeled the environment “Crawling”) and here are all the code lines I used:

from searchtweets import ResultStream, gen_rule_payload, load_credentials
import os

os.environ["SEARCHTWEETS_ACCOUNT_TYPE"] = "premium"
os.environ["SEARCHTWEETS_ENDPOINT"] = "https://api.twitter.com/1.1/tweets/search/fullarchive/Crawling.json"
os.environ["SEARCHTWEETS_CONSUMER_KEY"] = "<key>"
os.environ["SEARCHTWEETS_CONSUMER_SECRET"] = "<secret>"

premium_search_args = load_credentials("",yaml_key="")

rule=gen_rule_payload(pt_rule="#vaccine")
rs = ResultStream(rule_payload=rule,max_results=30,max_pages=1,**premium_search_args)
tweets = list(rs.stream())

I’m not sure if the problem is in the query itself or the setup of my authentication (for example the endpoint url?). Appreciate any answer.

[andypiper]

I just tested this same code with the Python 3 REPL locally, using my own consumer key and secret for the app that has access to the API, and I’m not seeing the same error. I can print(tweets) at the end of this code and get back the Tweet objects.

Do you have any other environment variables set in your shell, or are you only setting the values in the code?

Do you see these messages when the code is executed?

cannot read file
Error parsing YAML file; searching for valid environment variables
Grabbing bearer token from OAUTH

If your auth is setup correctly, I’d expect this to be output and to succeed. Otherwise, you’d likely see an error about your bearer token.

[miguelepdsilva]

Yes, I see that same message and when I try to print the “premium_search_args” the bearer token appears (according to the documentation of the package, they can hide the generation of the bearer token from users like me and we only need to give the API key).

I’m marking this question as resolved though. When I wrote this question, I had assumed that the function “gen_rule_payload” would automatically bind the number of results according to the account type. It turns out that since I’m using Premium Sandbox, I need to change it to explicitly only search the 100 results allowed, ie, changing the rule to: rule=gen_rule_payload(pt_rule="#vaccine",results_per_call=100).

Thank you and sorry to waste your time!

[andypiper]

Aha, great - yes, I had not spotted that difference between your invocation and mine. Thanks for clarifying the solution for others to learn from in the future.