POST to /statuses/update.json started hitting error 32 "Could not authenticate you." with no code changes

[schnarfed]

hi all! my app has been happily posting tweets via /1.1/statuses/update for a year or so. a week ago it started returning HTTP 401 {"errors":[{"code":32,"message":"Could not authenticate you."}]} for all of my users. i hadn’t changed any of my relevant code or app settings recently.

here’s the particularly odd part: when i try it in my local environment with the same code, twitter app key/secret, and even the exact same user access token key/secret, it works fine. it only fails in production.

i’m on google app engine using tweepy. (tried versions 2.2 and 3.3, the latest. both fail.) could app engine’s external facing IPs have been blacklisted or graylisted somehow?

more details in this github issue. any ideas? thanks in advance!

[andypiper]

Is there any chance that the time on your server has drifted or is inaccurate? this kind of thing could happen if the OAuth signature timestamps are off.

[schnarfed]

thanks for the quick reply @andypiper! and that’s a really great guess, i definitely wouldn’t have thought of it. unfortunately it’s not the issue here; i don’t personally control the server time (it’s a PaaS), but i know they run NTP, and i confirmed that the clock is accurate to within ~2s.

[kidneybingos]

Wondering if you’re having success with this. As far as I can tell, you are the only person having issues similar to mine. To wit:

1. I am running App Engine (Python). No issues in dev_appserver; only in production.
2. I am also using Tweepy and have tried both versions.
3. One difference from your issue, although it might not really be a difference if you have only tried POST requests, is that this is also happening on GET requests to the search API.
4. GET requests work fine most of the time, but if there is punctuation in my query (‘q’ parameter), most commonly an apostrophe, it always fails with the 32 “Could not authenticate you” error (but again, only in prod; punctuation causes no issue in dev).
5. It does not appear to be an encoding issue. I have verified that the apostrophe encodes to %27 as it should.
6. As far as I can tell, the encoded query string in production is exactly the same as on dev_appserver. Of course, I cannot compare the dev vs. prod raw URLs and headers sent to the API as apples-to-apples due to timestamp and nonce.
7. Nothing has changed in my code. These errors started occurring approximately 5 days ago. I send these queries hundreds of times a day and have been doing so for years.

It seems to be something that’s going on in the App Engine servers. I don’t think they’re being greylisted, due to the fact that my API calls do work if I leave out the punctuation. I’m wondering if perhaps the urlfetch service is mucking with the headers in some new way? Just a guess.

Let me know if you’ve made any progress on your end. Thanks!

[kidneybingos]

As some further (and strange) detail, it is not all or even most punctuation that causes this. As far as I can tell the full list of problematic characters is:

’ (apostrophe)
!
*
(
)

Similar characters such as brackets, braces, periods, commas, @ signs, etc. are all fine. I’m sure there are a couple others but I’ve tried lots and lots.

[djfische]

I’m seeing the same issue and it cropped up around the same time. I’m using Tweepy 2.3. I also confirmed that the Appengine system time is accurate within a couple seconds as well. I see no issues with the same code and tokens on the dev_server or running manually on the command line.

Have any of you guys tried without Tweepy?

[schnarfed]

glad to find some kindred spirits! just fyi, i’ve posted about this on the GAE google group.

[djfische]

Thanks for posting this but I don’t think Twitter has blacklisted Appengine’s servers. On @kidneybingo’s suggestion, I checked the content of my automated tweets. The tweets that were failing contained the “!” character. After removing that character from tweets, I’m seeing a 100% success rate. The tweets I was sending don’t contain the other problematic characters from @kidneybingo’s post so I can’t confirm them.

I’ll cross-post this to the Appengine group.

[waleedka]

Same issue here. Also started a little bit over a week ago. And also on Google App Engine.

• It happens when I call: /1.1/statuses/update_with_media.json but not when I call /1.1/statuses/update.json. Even with the same exact tweet.
• I get the error only if the tweet has a single quote or ! (I need to test the other characters that kidneybingos mentioned as well). If I remove the ’ and !, the posting works.
• If I repeat the same POST call several times, it often works on the 3rd or 5th attempt. Initially, I got the error about 5% of the time and our system keeps retrying so it wasn’t a major issue. But now the error happens in 90% of the calls and it’s becoming a big issue that our automatic retries can’t get around.

This is not the first time I’ve seen this. I posted about it back in December as well.

Authentication errors on tweets that have a single quote

This is a very odd error that started happening a few days ago. I post a tweet as usual (API v1.1) and it works most of the time, but occasionally it fails with: {"errors":[{"message":"Could not authenticate you","code":32}]} However: 1. The token is valid (I verified) 2. If I repeat the same exact request again, it works. So not an issue with the request formatting. 3. It never happens if the tweet is just simple text. I can publish 50 tweets and all of them work. 4. But if the post has a s…

[schnarfed]

to summarize the current state of affairs, we think app engine 1.9.20 introduced new URL-escaping behavior into the urlfetch API that sometimes breaks OAuth 1.1 signatures in Twitter API calls. we see this on 1.9.20 instances but not 1.9.19.

a google devrel person pointed us to this app engine issue, which sounds like the same thing. feel free to star it!

[schnarfed]

looks like app engine fixed their bug; this is no longer happening on 1.9.20. yay!

[andypiper]

Great news! Thanks for the detail and for keeping both communities updated on the issue!

[Nick22698553]

I know this is old, but not stale since I just came across this same issue. For me, the problem was the javascript algorithm I was using for percent encoding (i.e., encodeURIComponent) which was recommended in several blogs. That function does not comply with RFC 3986, Section 2.1 . so following is my fix

const percentEncode = ( inputStr ) => {
    let outputStr = encodeURIComponent( inputStr )
        .replace( /[!'()*]/g, c => '%' + c.charCodeAt(0).toString(16) );
    return outputStr;
};

[LeBraat]