I can’t connect Twitter API with NiFi GetTwitter 1.16.1 processor even though I have created an app with essential access and have all the required tokens and secret tokens. The error is: “Received error HTTP_ERROR: HTTP/1.1 403 Forbidden. Will attempt to reconnect”. Does anyone have any idea what is wrong?
It could be any number of things - do your credentials work in a different tool? Like twurl for example? GitHub - twitter/twurl: OAuth-enabled curl for the Twitter API A common thing that can happen is the credentials fail to be passed to the script because of wrong environment settings, so i would check how you’re configuring too.
Thank you for your reply. In twurl my credentials work fine, but I can’t tackle the issue with NiFi GetTwitter processor. Which settings should I check?
1 Like
I would double check to see if the credentials are definitely being loaded correctly (even try hard coding them maybe to test?)
I am encountering the same problem for two days. I am sure that the credentials are okay - 3 people checked that - created their own apps and tried to run NiFi GetTwitter account.
The problem is also that even simple Python script using Tweepy does not work with current credentials.
What might be the issue?
1 Like
The stream only allows 1 connection at a time, could the script be running somewhere else with the same credentials while you’re trying it at the same time? Or are these totally different developer accounts?
Yes, different accounts, different apps and machines. Tested on Ubuntu & Windows.
The weird thing is that this command works - using Bearer Token:
curl -X GET -H "Authorization: Bearer "<bearer_token>“Twitter_adress_to_specific_tweet”
On the other hand while trying to autenticate with API Key & API Key Secret & Access Token &Access Token Secret it does not work (and these are the ones needed in GetTwitter procesor).
Do you have any ideas?
import tweepy
api_key = “”
api_secrets = “”
access_token = “”
access_secret = “”
auth = tweepy.OAuthHandler(api_key,api_secrets)
auth.set_access_token(access_token,access_secret)
api = tweepy.API(auth)
try:
api.verify_credentials()
print(‘Successful Authentication’)
except:
print(‘Failed authentication’)
After running that code (with generated tokens form Twittter API) in Python the result is ‘Failed authentication’ - any ideas why?
Igor, Damian, thank you for your replies. Damian, it’s nice to know that I am not the only person that can’t access Twitter API with NiFi GetTwitter processor. However, the Python script you published probbly doesn’t work, because it uses old version of Twitter API (I am not sure, but I have encountered the same problem). You can find more informations here: Consumer Keys and Authentication Tokens doesn't work
1 Like
By the way, I still can’t find any solution, that will let me run NiFi GetTwitter processor. I’d appreciate any further information
After upgrading the account to ‘Elevate’ the Python code pasted above works now - unfortunately NiFi still does not - I would also much appreciate any help.
1 Like
Does NiFi thing work with a Bearer token only? Also check the server time if it’s in sync (it can affect oAuth)
The time is in sync between ubuntu and NiFi.
NiFi needs API Key & API Key Secret & Access Token & Access Token Secret.
Ok i just tried it, and GetTwitter 1.16.2 Processor in Nifi is actually using v1.1 API, not v2 API.
The ConsumeTwitter Processor that uses v2 it seems, is not released yet - nifi/org.apache.nifi.processor.Processor at main · apache/nifi · GitHub it’s in the latest master though, so it could be possible to run this from source yourself. But this part is something you’re better off getting help with from the Nifi devs.
If you DO have v1.1 Elevated access - make sure your pasted code is correct (I noticed it’s easy to add an extra space or newline when pasting the Keys into the properties in NiFi)
As I understand it works for you just fine?
I also use v1.1 Elevateted access and pasted the keys correctly - of that I am sure. Any more ideas or ideas worth looking into?
If you definitely have v1.1 Access, and the app is attached to an Elevated Access project, and if the keys are correct, the server time is in sync, and you can make calls with twurl, i’m out of ideas.
As an alternative, try twarc, twarc2 (en) - twarc streaming into a file - and then in Nifi read from that twarc output file instead of the GetTwitter Processor
I don’t if you guys found a solution I’m having the same problem
so I have Nifi 1.17.0 installed as docker
under my dev account when I click on overview I see 2 sections :
Elevated and the project I have created and the different keys , it has access to V1.1 Access and V2 Access
Stand alone project , with the keys I have generated , it has access to V1.1 Access only
I tried to use both set of keys but still same result I got 403
I’m trying to use getTwitter (Filter end point) but unfortunately no success I would appreciate if anyone gave an idea for me
Unfortunately even with Elevated Access, the v1.1 Sample Stream is now deprecated and no version of nifi supports v2 API yet - so you may have to implement an alternative
1 Like
Thank you for your reply this will avoid me another day searching
in case you have an idea , do you know any ope source alternative ?
My final need to intercept some terms and then send them to elasticsearch and I saw many people using NiFi as front end
If you can make your own connector, you can use twarc as a command line tool: twarc2 (en) - twarc writing to a file, with TailFile tailing an output file or TwitterAPI/stream_tweets_hydrate.py at master · geduldig/TwitterAPI · GitHub to feed elasticsearch directly or something along those lines.
1 Like
