I am a beginner and I am trying to collect some tweets from the API v2 for a University project (I would like just to create a corpus with tweets about a specific topic).
My account is “Standard Basic” and I already have a Project with a developer App attached.
But I am not able to download tweets because I get the HTTP Error code: 403
“When authenticating requests to the Twitter API v2 endpoints, you must use keys and tokens from a Twitter developer App that is attached to a Project. You can create a project via the developer portal.”
“title”:“Client Forbidden”,“required_enrollment”:“Standard Basic”,“reason”:“client-not-enrolled”
The code I am using:
search_tweets_v2:
endpoint: https://api.twitter.com/2/tweets/search/all
bearer_token: [my bearer token number]
search_args = load_credentials(filename="./twitter_keys.yml",
yaml_key="search_tweets_v2")
The file “twitter_keys.yml” has this configuration:
search_tweets_v2:
endpoint: https://api.twitter.com/2/tweets/search/all
bearer_token: [my bearer token number]
What could be the problem?
Thank you!
1 Like
Provided that you have everything set up in the dashboard https://developer.twitter.com/en/portal/projects-and-apps and it’s the correct bearer token from the correct app (bearer token contains a % so that sometimes fails to paste, or there’s an extra space or something else there that sometimes breaks it)
I would double check that you have the correct package:
pip uninstall -y searchtweets
pip install searchtweets-v2
And here’s a minimal working example that works for me (assumes that twitter_keys.yaml is in the same directory as this script):
import logging
import sys
logging.basicConfig(stream=sys.stdout, level=logging.INFO)
from searchtweets import load_credentials, gen_request_parameters, collect_results, ResultStream, write_result_stream
search_args = load_credentials(
filename="twitter_keys.yaml", yaml_key="search_tweets_v2", env_overwrite=False
)
query = gen_request_parameters("from:twitterdev", results_per_call=10, expansions="author_id", tweet_fields="created_at", start_time="2020-01-01", end_time="2020-01-31")
rs = ResultStream(request_parameters=query, **search_args)
stream = write_result_stream(rs, filename_prefix="test")
result = [tweet for tweet in stream]
logging.info(f"Made {len(result)} calls. Finished.")
1 Like
Thank you for your kind reply.
I have tried with your working example too, but I get the same error.
I also tried to regenerate the bearer token, but nothing has changed.
It looks like a problem with my developer account, even if I have a Standard Basic with V2 access.
To eliminate anything to do with python / environments - replacing AAA...zzz and running this should give you a result:
curl -H "Authorization: Bearer AAA...zzz" "https://api.twitter.com/2/tweets/search/recent?query=from%3Atwitterdev%20-is%3Aretweet&max_results=10"
If not, then there is definitely something wrong with the token.
The App that you have, has to be added to the Project for it to work (it should be under “Standard” not under “Standalone Apps”).
Unfortunately it does not work.
Yes, the Project and the App attached are under “Standard”.
What can I do now? 
Thank you again for your help
Can you make v1.1 Calls with the same Bearer token? Also is the bearer token definitely copied properly? it should look like:
AAAAAAAAAAAAAAAAAAAAAAAsdfgsAAAAvSDFGSDRgssdfSDFGSDF44gsd4E%3Dkk33345336dfsgsdgsdgsdASGASDGadsGAFAKJGYIUYUIDGGKK
This one is fake but it’s the right length and format - if your one is shorter it’s probably not copied correctly.
Does this work?
curl -H "Authorization: Bearer AAA...replace with token...KK" "https://api.twitter.com/1.1/users/show.json?screen_name=TwitterDev"
(you should get some json back)
Is your system time in sync? Sometimes it can interfere with oAuth
That’s the only thing i can think of to try.
My Bearer Token has the same lenght, more or less, with three %.
My original code (suggested from my professors and that works for them) was:
!pip install searchtweets-v2
from searchtweets import ResultStream, gen_request_parameters, load_credentials, collect_results
import csv
import os
search_args = load_credentials(filename="./twitter_keys.yaml",
yaml_key="search_tweets_v2")
arguments = "...keyword chosen... -is:retweet -is:quote lang:it"
metadata = ["created_at", "author_id", "source"]
query = gen_request_parameters(arguments,results_per_call=100, tweet_fields=",".join(metadata))
rs = ResultStream(request_parameters=query,
tweetify=False,
max_results=1000,
max_pages=1,
**search_args)
empty = not os.path.isfile("corpus.csv")
with open ("corpus.csv", "a") as output_file:
writer = csv.DictWriter(output_file, fieldnames=["id", "text"]+metadata)
if empty:
writer.writeheader()
for tweet in rs.stream():
if "id" in tweet:
writer.writerow(tweet)
With the code written above (with the same token) and v1.1 access attempt, I get another error:
> HTTP Error code: 404: {"errors":[{"code":50,"message":"User not found."}]} | Not Found
I have tried to past the code you gave me:
curl -H “Authorization: Bearer AAA…replace with token…KK” “https://api.twitter.com/1.1/users/show.json?screen_name=TwitterDev”
And it gives me a “SyntaxError”, but it is possible that I have pasted it in the wrong place (I have a really poor knowledge of programming, sorry).
Are you running this in a notebook? Try running it in a terminal / command prompt.
Another thing to try is maybe twarc - there are some good links to tutorials here:
In the command prompt, it works!
Even if not in the .csv format, I have been able to collect a lot of tweets 
I can collect data using Twarc too.
Thank you!
I was using the code I wrote before on Google Colab because I am not so familiar with Python.
I don’t understand why it works for my professors but not for me.
1 Like
Oh, twarc can also do this:
Install the plugin:
pip install twarc-csv
Now there’s a new command:
twarc2 csv results.jsonl tweets.csv
(if you saved your tweets from twarc2 search command and called it results.jsonl)
Thank you so much!
Two more questions:
-With Twarc 2 “search” command, how can I put a start time and an end time?
I have found this
twarc2 search '(topic)' --archive --start-time 2020-05-10 --end-time 2020-05-20 > tweets.json
But the result is “Client Forbidden”
-For the “stream” option, I tried with
twarc2 stream-rules add "word"
twarc2 stream --limit 1000 output.jsonl
But I get the error “Connection broken: InvalidChunkLength”.
Thank you again for your support!
This error appears when you try to use --archive (that uses a different endpoint). This only works for academic access projects (your dashboard should have a separate “Academic Access” project with an app attached, not “Standard”) Twitter Developer Access - twarc
That’s the correct way, but i would suggest a slight modification:
twarc2 search "query" --archive --start-time "2020-05-10" --end-time "2020-05-20" tweets.json
to make things more compatible in different terminals, it’s always better to specify tweets.json as the last parameter, and let twarc handle writing the file instead of an output redirect > that means the terminal has to write the file.
“Connection broken: InvalidChunkLength”
This error i haven’t seen before - maybe it’s related to something that came up with the stream where it would break after 5 minutes of low activity. It would help to check twarc.log file for any clues- but in this case i would restart the stream and see if it keeps happening. I have a suspicionthe error is from twitter’s end - sending an empty piece of data instead of a newline.
same problem here, except that I have an active account indicated by:
However still getting 403 forbidden error to connect with Twitter API.
What exactly are you running to get the 403 error?
I created a client to grab tweets for me.
Authentication auth = new OAuth1(CONSUMER_KEY, CONSUMER_SECRET, TOKEN, SECRET);
ClientBuilder builder = new ClientBuilder()
.name("Hosebird-Client-01")
.hosts(hosebirdHosts)
.authentication(auth)
.endpoint(hosebirdEndpoint)
.processor(new StringDelimitedProcessor(msgQueue));
Client client = builder.build();
Ah, it’s maybe worth creating a separate topic for this, as this is a different implementation.
As far as i remember, hosebird client does not support v2 API. And you need Elevated Access to use v1.1 APIs that hosebird client connects to.
For v2, I still recommend using twarc as a command line tool: https://twarc-project.readthedocs.io/
For java / hose bird client, using v1.1 API definitely worth making a new, separate topic, if that’s really what you need to get working.
Thanks Igor!! Do you know how to elevate access from hbc client to v1 api? Or is there a guidance link on this?
To get Elevated Access, there should be something in the dashboard for it, I don’t have an exact link because it doesn’t appear to me: https://developer.twitter.com/en/portal/dashboard maybe on the app settings
