OAuth still returns 403 (error 415) even when callback urls are set

[warblealerts]

Hi there everyone

Became aware of the changes of oauth around the 12th due to production errors. Have since added all callback URLs to the app settings console but still unable to “Sign in with Twitter” and get 403:

<?xml version="1.0" encoding="UTF-8"?>Callback URL not approved for this client application. Approved callback URLs can be adjusted in your application settings

I’ve checked, checked again, checked once more and tried different things but I just cannot get it to work. Any pointers much appreciated.

Many thanks!

[warblealerts]

Hi there again

Unfortunately, we still have this problem.

The Twitter API just doesn’t accept our whitelisted call back URLs - in dev or production.

Is there a possibility there is an issue on the Twitter API side here? I’ve debugged the oAuth connection locally to verify the callback url is correct and whitelisted in app settings, the base path, the calling path, but I continue to receive 403 errors.

Appreciate any help with this issue.

Thanks

[andypiper]

What’s the specific URL path in your application that is listening for the OAuth callback for Twitter?

You need to add your fully-qualified domain, plus that path, minus any parameters, into the Callback URL panel on apps.twitter.com (feel free to share a screencap if you like).

So let’s say I have a webapp running on my server myserver.co.uk, and it has a callable path for receiving user account tokens from Twitter at /twitter/auth - we want to configure Twitter to call us back there on successful completion of authentication.

So the callback url would be https://myserver.co.uk/twitter/auth. We also need to make sure that in our code, when we call oauth/request_token, we provide that exact same value in the callback_url parameter.

[warblealerts]

Hi there @andypiper

Our callback URL is /auth/twitter/callback.

This is defined in the apps panel as:

https://warble.co/auth/twitter/callback
https://warble.co/auth/twitter

Our users still can’t sign in and I don’t really know what else I can try to get this working again.

Is there a possibility here that something is wrong on the twitter side for our app? In the absence of a more detailed error or diagnostics from Twitter for the failure, how can we try and resolve this? Can we raise a ticket with Twitter dev support for more insight as to why our whitelisted URLs are not being accepted?

Many thanks for you help.

[LeBraat]

Before we escalate, I want to verify a couple of things:

1. You have both https://warble.co/auth/twitter/callback and https://warble.co/auth/twitter set up in your apps dashboard.
2. You are using one or both of the above URLs with the POST oauth/request_token endpoint.

[warblealerts]

Thanks for coming back. Sure.

1. Yes - confirmed

2. Yes - you can see the endpoint called in the logging below

I have created a bare bones test app but with our Twitter keys and I still encountered the same problem in development.

I added httplog gem to see some basic tracing in the rails app. I then set the client_options.site parameter available in omniauth-twitter to point the oauth calls to a simple python server, to trace out the initial POST request headers.

Here are the results (sensitive keys removed):

I, [2018-06-24T12:51:54.196417 #13208]  INFO -- omniauth: (twitter) Request phase initiated.
D, [2018-06-24T12:51:54.198424 #13208] DEBUG -- : [httplog] Connecting: api.twitter.com:443
D, [2018-06-24T12:51:54.243109 #13208] DEBUG -- : [httplog] Sending: POST http://api.twitter.com:443/oauth/request_token
D, [2018-06-24T12:51:54.243258 #13208] DEBUG -- : [httplog] Data: 
D, [2018-06-24T12:51:54.376164 #13208] DEBUG -- : [httplog] Status: 403
D, [2018-06-24T12:51:54.376230 #13208] DEBUG -- : [httplog] Benchmark: 0.132842 seconds
D, [2018-06-24T12:51:54.376288 #13208] DEBUG -- : [httplog] Response:
<?xml version="1.0" encoding="UTF-8"?><errors><error code="415">Callback URL not approved for this client application. Approved callback URLs can be adjusted in your application settings</error></errors>

incomming http:  /oauth/request_token
127.0.0.1 - - [24/Jun/2018 12:40:16] "POST /oauth/request_token HTTP/1.1" 200 -
ERROR:root:Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
Accept: */*
User-Agent: OAuth gem v0.5.4
Content-Length: 0
Content-Type: application/x-www-form-urlencoded
Authorization: OAuth oauth_callback="http%3A%2F%2Flocalhost%3A3000%2Fusers%2Fauth%2Ftwitter%2Fcallback", oauth_consumer_key="keykeykeykeykeykey", oauth_nonce="qWUl5mImPZ0vqH4AGjkwt7rU2aX8YpZ8y0TUBLLYn8", oauth_signature="sigsigsigsigsigsig, oauth_signature_method="HMAC-SHA1", oauth_timestamp="1529840416", oauth_version="1.0"
Connection: close
Host: api.twitter.com:8000

So I can see/confirm that my callback url is:
http://localhost:3000/users/auth/twitter/callback

Of course, I’ve added the following urls to app settings:

http://localhost:3000/
http://localhost:3000/users/auth/twitter/callback
http://localhost:3000/users/auth/twitter

And still the 403 forbidden error - the same problem we have on our production site.

Many thanks

[davidegi]

Hello there, did you try to generate again the Twitter app keys? We had the same issue fixed reissuing the keys.

Davide

[LeBraat]

You can not set up your callback URL as Localhost. You can read more about this in this existing topic:

Callback URL not approved - but worked for months

I did not use the callback url. I typed https://www.google.com in the callback url. This is a usage posted on the twitter-kit-unity site. I developed the unity app and logged into twitter with the twitter-kit-unity plugin. So far, it worked fine. Is there any problem with entering callback url anything? I do not know which callback url to use. I do not have a callback url. Do I have to make this? Please provide detailed manuals. This process is difficult for users who simply use the Twitter S…

[warblealerts]

Hi @LeBraat

Were you able to escalate for us on the production domain - warble.co?

Thanks

[warblealerts]

Hi @davidegi

If we do this will it break all of the current access tokens we have?

Thanks

[warblealerts]

Is anyone there able to help us investigate and identify why our production app will not work with the correct whitelist?

Thank you!

[LeBraat]

Hey @warblealerts,

Please make sure to review our callback URL page on our docs. We made some updates to it recently to better explain what callback URLs will work with our system.

Then, please use the same callback URL in both the whitelist section of the apps dashboard and with your oauth/request_token request.

Once you have done this, if you are still running into issues, please send me the callback URLs that you are using, the request that you are making to the oauth/request_token endpoint, and the error message that you are receiving.

[MNedokushev]

Hello.

We also have the same issue in one of our applications.
That’s our current whitelist:

    http://megogo.net
    https://cms.qa.megogo.net/new_auth/diag/twitter
    https://megogo.ru/new_auth/diag/twitter
    https://cms.rc.megogo.net/new_auth/diag/twitter
    https://megogo.net/new_auth/diag/twitter
    https://rc.megogo.ru/new_auth/diag/twitter
    https://embed.qa.megogo.net/auth/social_dialog2/twitter
    https://embed.rc.megogo.net/auth/social_dialog2/twitter
    https://embed.megogo.net/auth/social_dialog2/twitter

An application that is not working is https://embed.megogo.net/ (embed.rc and embed.qa do not work too). Also, there is one thing to consider - the application https://megogo.net that you can see in our whitelist works fine. It’s strange and I can’t see any difference between them.

Ok, that’s the request to /oauth/request_token:

POST https://api.twitter.com/oauth/request_token
Headers: "Authorization" -> "OAuth oauth_nonce="667926019", oauth_signature="EbqFSmbmTYB54sZ03s0HNAjRyTY%3D", oauth_callback="https%3A%2F%2Fembedv3-dev.qa.megogo.net%3A9007%2Fauth%2Fsocial_dialog2%2Ftwitter", oauth_consumer_key="9nWKxrdoKtcy4lhqtfIiNw", oauth_timestamp="1535982330", oauth_signature_method="HMAC-SHA1", oauth_version="1.0""

Note: I got this data from my local instance of an application, so you can see a domain that is not in the whitelist in oauth_callback parameter. I hope that’s okay and doesn’t make any difference for your investigation.

Error message: “OAuthException: Response body is incorrect. Can’t extract token and secret from this: ‘<?xml version=“1.0” encoding=“UTF-8”?><errors><error code=“415”>Callback URL not approved for this client application. Approved callback URLs can be adjusted in your application settings</error></errors>’”

[abraham]

The oauth_callback value of your OAuth request contains the value "https://embedv3-dev.qa.megogo.net:9007/auth/social_dialog2/twitter"which is not in your list of alllowed callback URLs. Protocol, subdomain, domain, port, and path all need to match exactly.

[MNedokushev]

In my post I wrote the following:

Note: I got this data from my local instance of an application, so you can see a domain that is not in the whitelist in oauth_callback parameter.

[warblealerts]

Hey @LeBraat

Thanks for coming back - much appreciated.

Following your last message, I took a step back and searched for an “example app” in the Ruby world. I wanted to to see if, when using our keys, it would work. I found a good example app and tried it with our keys and it DOES work!

However, I still cannot get our main app working, even with the benefit of the example app and the source code. For some reason our main app does not get past the Request phase, failing with 403 (Callback URL not approved for this client application. Approved callback URLs can be adjusted in your application settings). I’ve used the same setup for dev that I know works on the example apps.

Even though we now know the whitelist is working, I believe something has changed on the twitter side, because we hadn’t made any changes to our code for a while and then it suddenly stopped working when the whitelist change was introduced - whatever changed broke the existing code/gems we are using. I know our app wasn’t whitelisted correctly when the whitelisting change was introduced, but even once that was done it still didn’t work.

I’ve monkey patched the oauth gem’s calls on non-working/working apps to trace out the parameters but they look identical. Maybe the problem is deeper down in the other dependent libraries a weird interaction between gems. I will continue to look at this in the meantime.

Is it possible for you to look at the logs for our (dev/production) oauth calls and see why they are failing with 403? What specifically is the problem - is it a parameter or something else with the call that is wrong? Having a little more to go on would be really helpful in trying to locate the library that is causing the problem. Any additional insight would be valuable. Please PM the results if that is safest.

If this problem is affecting me then perhaps it is affecting others using the same gem. If we can locate the problem then we can share that insight with the ruby community.

Thanks!

[warblealerts]

Hey @LeBraat

After many painful hours we have finally figured out the cause of our issue!

I’ve checked our repo and we have been using our warble alerts user consumer/secret with OAuth and not the application’s consumer/secret.

I guess historically this has always worked (or defaulted to the primary/first app), but coincidentally got changed on the Twitter side around the time of the whitelisting release. I think the timing and the fact our whitelisting URLs were wrong just pointed to that being the problem initially and us then searching for a problem/resolution around that.

Deeper monkey patching on OAuth tonight, tracing out the authentication header, showed up the difference between “good” and “bad” apps, exposing the incorrect key usage.

I just wanted to thank you for your help and I hope we haven’t wasted too much of your time as I know you guys are very busy right now.

Cheers

[warblealerts]

Actually, one final thought:

Maybe a different error message could be shown when a “Sign in with Twitter” is attempted with an consumer/secret that is not an actual app API consumer/secret. e.g. The key used is not registered with an app.

The error message suggests it is a whitelisting problem when it is a key problem.

[LeBraat]

Thank you very much for the suggestion. I will make sure to communicate this to our product team.

[LeBraat]