Why doesnt twitter verify the HMAC-SHA1 signature on /oauth/access_token?

[not_me]

Aside from the fact that I’m routinely finding that following the specs in the API documentation is the quickest way for you to get your oauth query to fail, a bug in my code allowed me to discover that Twitter is not validating the HMAC-SHA1 signature in at least the API calls to /oauth/access_token. This happened when I ‘accidentally’ supplied the oauth_verifier parameter to the API, causing the request to return a 401, checked a libraries output and saw that it supplied the oauth_token I was returned in the call to /oauth/request_token instead and forgot to remove the verifier parameter when I submitted my reformed request.

This is a security bug and needs to be escalated appropriately.

My code to generate the request is as follows, take particular note of the lines:

	oaStr += QUrl::toPercentEncoding("&") + QUrl::toPercentEncoding("SCREW_YOU=TWITTER_Y_U_NO_CHECK_THE_SIGNATURE");
	m_dmap["oauth_signature"] = hmac_sha1(m_dmap["oauth_consumer_secret"].toAscii(), oaStr.toAscii());

Full function:

bool
oauthAccessToken_t::doRequest(void)
{
	QString oaStr("POST&");
	std::map<QString, QString> oaMap;

	oaStr += QUrl::toPercentEncoding(m_urls["access"]);
	oaStr += "&";

	m_request->setUrl(QUrl(m_urls["access"]));

	
	oaMap["oauth_nonce"]			= m_dmap["oauth_nonce"];
	oaMap["oauth_signature_method"] = m_dmap["oauth_signature_method"]; 
	oaMap["oauth_timestamp"]		= generateTimeStamp();
	oaMap["oauth_consumer_key"]		= m_dmap["oauth_consumer_key"];
	oaMap["oauth_version"]			= m_dmap["oauth_version"];

	for (auto itr = oaMap.begin(); itr != oaMap.end(); itr++) {
		oaStr += itr->first + QUrl::toPercentEncoding("=") + QUrl::toPercentEncoding(itr->second);
		oaStr += QUrl::toPercentEncoding("&");
	}

	oaStr = oaStr.mid(0, oaStr.length()-3);
	oaStr += QUrl::toPercentEncoding("&") + QUrl::toPercentEncoding("SCREW_YOU=TWITTER_Y_U_NO_CHECK_THE_SIGNATURE");
	m_dmap["oauth_signature"] = hmac_sha1(m_dmap["oauth_consumer_secret"].toAscii(), oaStr.toAscii());
	
	m_dmap["oauth_hdr"] =	"OAuth oauth_consumer_key=\"";
	m_dmap["oauth_hdr"] += oaMap["oauth_consumer_key"] + "\", oauth_nonce=\"";
	m_dmap["oauth_hdr"] += oaMap["oauth_nonce"] + "\", oauth_signature=\"";
	m_dmap["oauth_hdr"] += QUrl::toPercentEncoding(m_dmap["oauth_signature"]) + "\", oauth_signature_method=\"";
	m_dmap["oauth_hdr"] += oaMap["oauth_signature_method"] + "\", oauth_timestamp=\"";
	m_dmap["oauth_hdr"] += oaMap["oauth_timestamp"] + "\", oauth_token=\"";
	m_dmap["oauth_hdr"] += m_dmap["oauth_token"] + "\", oauth_version=\"";
	m_dmap["oauth_hdr"] += m_dmap["oauth_version"] + "\"";
	

	m_request->setHeader(QNetworkRequest::ContentTypeHeader, "application/x-www-form-urlencoded");
	m_request->setRawHeader("Authorization", m_dmap["oauth_hdr"].toAscii());
			
	m_reply = m_mgr->post(*m_request, QByteArray());
	this->connect(m_reply, SIGNAL(finished()), this, SLOT(gotReply()));
	return true;
}

As noted, this request, despite having both invalid parameters per https://dev.twitter.com/docs/api/1/post/oauth/access_token AND and invalid signature due to the spurious string I elected to include, is perfectly Okay per the API. I’ve not edited out my consumer key for this request, it’s a non-issue in the first place, but I have since regenerated a new set; I have however edited out the session id and similar in the cookie just for paranoia’s sake. The expected behavior is that Twitter responds with a 401 because my signature is NOT valid, however it tells me everything is okay and gives me an access token and such.

POST /oauth/access_token HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Authorization: OAuth oauth_consumer_key="8BpuepyGkukZn8R0ynXHMg", oauth_nonce="VDXVDZXFQICOHEOZYZZZESZZYCMUUHDA", oauth_signature="wtUbvRUmVyGLUzp57DxFmqKzshg%3D", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1342776647", oauth_token="rgO9ZGzeQ2x5MlEk1G42B2qvwX5wrPaWK6LRq3Dzw3E", oauth_version="1.0"
Cookie: k=10.35.21.138.1342776625210251; guest_id=v1%3A134277662522051888; _twitter_sess=BAh7CCIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxhc2g6OkZsYXNo%250ASGFzaHsABjoKQHVzZWR7ADoPY3JlYXRlZF9hdGwrCEVIuaM4AToHaWQiJWY1%250AOTIxOTRjZDQzNWYyMjQ1MTI3NmE2M2QyODJjOGY3--1055b0a72f3f051cc8e30bce299dde2d171360b2
Content-Length: 0
Connection: Keep-Alive
Accept-Encoding: gzip
Accept-Language: en-US,*
User-Agent: Mozilla/5.0
Host: api.twitter.com

HTTP/1.1 200 OK
Date: Fri, 20 Jul 2012 09:30:56 GMT
Status: 200 OK
Pragma: no-cache
ETag: "3196adf196be1951f7ea5e370d57fc7a"
Expires: Tue, 31 Mar 1981 05:00:00 GMT
Last-Modified: Fri, 20 Jul 2012 09:30:56 GMT
X-Runtime: 0.07855
X-MID: 815222e1ce1d54d567e52347bceb4c839229a53d
Content-Type: text/html; charset=utf-8
X-Frame-Options: SAMEORIGIN
X-Transaction: fcaef40d58eb101e
Cache-Control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 160
Server: tfe
[...]

As I said previously, this is a security bug and needs to be treated as such. More over, my expectation is that other aspects of the API do not validate the signature as they should; if I had to take a wild stab at it, I would guess everything but /oauth/request_token probably ignores the signature as some form of misguided optimization.

[froginthevalley]

Thanks for reporting this, we are investigating the issue.

[not_me]

Hi,

Firstly, all apologies for the sorta snarky posting, mostly venting frustration with the API; attacking twitter is not my goal, I was in the process of writing a legitimate client.

That said, upon getting some sleep and thinking some more, I think the signature is a symptom of the problem and less the problem itself, albeit it’s still a problem. It’s lessened by the fact that the consumer secret and consumer key are not really secret for any application that ships a binary to the end user, for example, here are the consumer key & secret for tweetdeck: http://twitpic.com/a9t105.

There are essentially 3 colliding issues here; the first is that I can obtain the keys for such clients and thus sign requests for the beginning steps of the process. This is not twitter specific and will hold true for oauth in general.

The second is that the callback URL is entirely unused; as I noted about the oauth_verifier parameter is unused, I could point the URL to google and it wouldn’t matter at all. This drastically reduces the complexity for a potential attack and places all of the security on the token thats obtained as the result of the request token API call, which again, anyone with access to the binary can do.

These tokens are not replayable in the sense that after a user has authorized an application and after the app has requested an access token I cannot re-use the request token to receive another access token. However, there is a race condition here in that anyone with the consumer key can make guesses at what a valid oauth_token was and if right and if you hit the window between when the user authorizes the application but before the app requests the authorization token then they will receive the access token instead of the legitimate 3rd party app. If the oauth_verifier parameter was requisite, this would greatly increase the security.

Finally, the request tokens themselves do not appear to be formed via an overly strong PRNG (assuming its actually ‘random’). Over the course of today I requested 41,130 request tokens and performed phase state analysis (references below) to test the randomness of the tokens returned; assuming there are no bugs in my code used to generate the data, then this graph (http://twitpic.com/a9yrt1) is fairly startling in that the tokens appear to be fairly predictable and suggesting that being able to obtain access tokens for random accounts is a very real possibility. The graph should show dots essentially all over the space, however they show a very striking line that correlates to time, with breaks in the line actually being places where I shutdown my computers and stopped acquiring tokens.

The fix, so to speak, is fairly simple, the reality is going to be less so I imagine:

• incorporate and require the oauth_verifier parameter
• increase entropy in the request tokens
• validate the signature on the requests

The downside is that I imagine it will break virtually every client utilizing the api. o_O

There are some more tests I need to run to make sure I don’t have a bug in my code (or far more likely somewhere in the BigInt library I am using) anywhere in terms of the phase state analysis, and I will post back tomorrow after I’ve had a chance to do that.

‘A Phase Space Analysis of Network Protocols’ http://www.ida.liu.se/~TDDD17/oldprojects/2007/projects/9.pdf

'Phase Space Analysis of Session Cookies’

9.pdf

'Strange Attractors and TCP/IP Sequence Number Analysis’
http://lcamtuf.coredump.cx/oldtcp/tcpseq.html

Cheers.

[not_me]

So yes, I had a bug in my code for generating the graphs, however with that fixed things are not any better.

Here is a graph of not really random data:

not really random

Here is a graph of seconds / milliseconds:

seconds / milliseconds

Here is a graph of rather random data:

quite random

Here is a graph of 200,007 request tokens received from twitters servers over this past weekend:

This is with 200,007 request tokens in the sample set.

[mohteshim_ahmad]

The official Mohteshim Ahmad Indian account…