Redirect URL receiving state parameter with some characters stripped out

[ethanlee]

The state parameter passed to the OAuth authorization page (/i/oauth2/authorize) seems to get some characters stripped out upon a successful redirect.

These include URL-encoded quotes (%22) and URL-encoded commas (%2C).

Example case

When providing a URL-encoded state parameter representing JSON to /i/oauth2/authorize like the following:

state=%7B%22some%22%3A%22object%22%2C%22with%22%3A%22string%20values%22%7D

(This can be generated with this JS)

encodeURIComponent(JSON.stringify({ some: 'object', with: 'string values' }))

Upon redirect, the received URL state parameter is:

state=%7Bsome%3Aobjectwith%3Astring+values%7D

When decoded, we see that the quote and comma characters have been removed:

{some:objectwith:string+values}

[gauloics]

Welcome yo the community

Hi @ethanlee Just wondering, why are you using objects, while in the documentation state values ​​simply use random strings

@ethanlee

{ some: 'object', with: 'string values' }

If I use the online encode tool the result is different from the encodeURIComponent(JSON.stringify()) result

This is the result of the online tool:

%7B%20some%3A%20%27object%27%2C%20with%3A%20%27string%20values%27%20%7D

And if decoded the result is exactly the same as
{ some: 'object', with: 'string values' }

[ethanlee]

Thanks for the reply @gauloics – while the Twitter-specific docs may suggest using random values for state, that is up to our discretion. According to the RFC:

state: An opaque value used by the client to maintain state between the request and callback. The authorization server includes this value when redirecting the user-agent back to the client. The parameter SHOULD be used for preventing cross-site request forgery as described in Section 10.12.

We do use the state parameter in our use case to maintain state between request and callback: partly for preventing CSRF and partly for other purposes.

---

@gauloics

If I use the online encode tool the result is different from the encodeURIComponent(JSON.stringify()) result

For the online encode tool, you’ll need to use the JSON-stringified representation of the object we’re using:
{"some":"object","with":"string values"}.

But the difference you’re seeing between your decoded result (bottom) and the value we’re getting back from Twitter (top) is exactly the issue we’re encountering.

+ state=%7Bsome%3Aobjectwith%3Astring+values%7D
- state=%7B%22some%22%3A%22object%22%2C%22with%22%3A%22string%20values%22%7D

Twitter’s service is providing us the top (green) value after the callback, unexpectedly stripping out characters when passing in the bottom value to the original authorization URL.

[gauloics]

@ethanlee

But the difference you’re seeing between your decoded result (bottom) and the value we’re getting back from Twitter (top) is exactly the issue we’re encountering

Ah my bad, miss checking

[ethanlee]

Any ideas why this might be happening?

We are not keen on using a different encoding for the state parameter as a specific workaround to Twitter’s OAuth service. For every other OAuth provider we’ve implemented, this has been no issue at all.

It’s especially puzzling that the URL-encoded characters are being filtered out – is this not the standard encoding to make these characters URL-safe?

[gauloics]

Twitter staff usually reply within a few days, and it looks like this hasn’t been reported before, hope they respond quickly

[alanbenlee]

Hey @ethanlee, thanks for reaching out. I haven’t seen this happen before but I’ll be doing some testing on to see if I can replicate the issue.

[ethanlee]

Hi @alanbenlee, thanks for the reply! If it’s helpful, I put together this Glitch project to demonstrate the issue: https://twitter-oauth-redirect.glitch.me/

Notice that for simple strings, it works as expected (pass in 12345, receive back 12345).

But then when trying something in object notation: {"test":"do quotes work too?"}, the result is:

(the quotes are gone!)

If you’d like to try it on your own Twitter client ID and Glitch project, feel free to remix it and change the env var TW_CLIENT_ID. This project doesn’t actually perform the last code/token exchange, so it only generates authorization codes (which are unused).

[ethanlee]

Wondering if anyone has gotten a chance to reproduce the issue, with the above Glitch project.

We are still affected by this issue, and this is the only OAuth 2.0 provider that we’ve worked with so far that exhibits this incorrect character handling. My best guess is that this is some overeager prevention of SQL injection internally?

[IgorBrigadir]

Would a potential workaround for this be if you base64 encode it?

[ethanlee]

Yes, thanks for the suggestion!

This would be a workaround, and we’ve considered it on our team, but we’re not interested in changing our current OAuth process at this time only to support this specific issue we see with Twitter. We integrate with a number of providers and have not seen this issue with any other service.

I just thought that I would raise this issue for the API team so that they understand that this is a bug with their OAuth implementation.

[ethanlee]

@alanbenlee Hi, just checking in to see if the demo above was helpful in replicating the issue. Let me know if I can provide other info. Thanks!

[ethanlee]

Hi! Just wanted to bump this thread – don’t want it to accidentally close without resolution. Thanks!

[ethanlee]

Another bump on this thread!

[ethanlee]

Bump!

[_gabe]

Thanks for sharing your situation.
I have a question, is the state parameter the only way we have to pass data inside the authorization url so the backend hosted as the callback url can receive that data? I’ve tried using query params inside the callback url but then I get an error of callback url mismatch when trying to exchange the authorization code for a token:

{"error":"invalid_request","error_description":"Value passed for the redirect uri did not match the uri of the authorization code."}

It works when I discard the parameters