Using * in search query not allowed?

ebshih · 2012-12-19T23:25:46.000Z

Does twitter not allow using *'s as part of the value of the q term for the users/search API? Testing on the developer console seems to have no problem but within an app I am running I am getting the following error anytime I add * to the search string: for example “Bob * Smith”

ApplicationName doesn’t have permission to access your Twitter account anymore. Click here to sign out of Twitter, then sign in again to restore access permissions for ApplicationName.

Inspecting the code, the request URL being sent is “https://api.twitter.com/1.1/users/search.json?count=20&q=Bob+*+Smith”.

ebshih · 2012-12-19T23:41:58.000Z

btw I left out the oAuthtoken and oauthsecret and request uuid on purpose

ebshih · 2012-12-20T00:16:13.000Z

I am getting a 401 error {“errors”:[{“message”:“Could not authenticate you”,“code”:32}]}, but if I search without the *, e.g. Bob Smith, I get the results no problem.

episod · 2012-12-20T16:07:26.000Z

The asterisk “*” character is a reserved character – you’ll need to encode it as “%2A” for usage in URLs and POST bodies. In the OAuth signature base string that means you’ll have it encoded once more after that.

ebshih · 2012-12-20T18:33:00.000Z

I’m using the Play library’s WS.encode(), My understanding is that they do the double encoding but are you suggesting that the encoding is the issue and if the * isn’t being encoded right by Play then Twitter sends a 401? Why is it a 401 and not an invalid format 406 or something? That doesn’t make sense to me.

Also, how come when I try to add Bob * Smith as the query in the developer console it generates the GET URL as http://api.twitter.com/1/users/search.json?q=Bob%20*%20Smith, and the * doesn’t look encoded to me. I also noticed the 1 in the URL and not 1.1. Is the developer console just out of date?

Play API for WS.encode(): http://www.playframework.org/documentation/api/1.2.5/play/libs/WS.html#encode(java.lang.String)

ebshih · 2012-12-20T23:34:00.000Z

Are you saying that if the * is not encoded properly then the API returns an authentication error 401? It seems to me that it should be something like a 406 of invalid format of request or some sort of bad input error if that were the case. Also, I am just using the Play library’s WS.encode() to encode the search string, which I haven’t read any issues about.

Also, from the twitter developer console, if I type the search string Bob * Smith, it gives the GET UR with query string q=Bob%20*%20Smith, but it looks like it is using API v1. Is this a new change to v1.1?

episod · 2012-12-21T15:23:55.000Z

Not all URL escape libraries do the right things by default. Many don’t consider characters such as “*” and “:” to be reserved characters when they in fact are – or only consider them reserved characters when you’re explicit about them appearing in a parameter versus as part of the path.

We use HTTP 400 Bad Request when your request is malformed enough that our HTTP servers cannot understand the request.

We can understand your request in this case, and have passed it on to the authentication service.

The reason it’s likely a HTTP 401 is because our auth filter can’t verify your authentication due to not being able to recreate a valid signature for you – it’s following the strict OAuth spec in evaluating your request and when it generates an OAuth signature given your inputs the result is different than what you came up with.

API v1 has some endpoints which are as strict as this, but the majority are more lenient. All of API v1.1 is strict on this point.

ebshih · 2012-12-21T20:59:00.000Z

Are you sure that the encoding of the * is the issue? I just tried adding another layer of encoding to encode the * as %2A, and to encode spaces as %20 instead of +, but I am still getting the 401.

The GET URL being sent is now:
https://api.twitter.com/1.1/users/search.json?count=20&q=Bob%20*%20Smith

Once again, using the same oauth token and consumer key, a request encoded in the exact same way has no problem:
https://api.twitter.com/1.1/users/search.json?count=20&q=Bob%20Smith

ebshih · 2012-12-26T18:17:09.000Z

I’m going to try and post this again, since it keeps queuing my responses and I need this to be resolved ASAP.

@Taylor, Are you sure that the escaping is the issue causing the 401 error? I tried adding an extra layer of encoding that encoded * as %2A but I am still getting the error.

The query string for the GET request is now Bob%20%2A%20Smith, but I still get a 401.
With the exact same oauth token/secret and consumer key/secret, a request with the exact same encoding for Bob%20Smith works perfectly fine.

ebshih · 2012-12-27T23:46:10.000Z

bump. @episod

episod · 2013-01-02T15:46:44.000Z

Sorry for the delay in response over the holidays.

I’m not a 100% sure that your issue is with the “*” character encoding.

I do know that I’m able to successfully perform this request by ensuring the character is encoded in the URL and appropriately escaping in the signature basestring:

URL:
https://api.twitter.com/1.1/users/search.json?count=20&q=Bob%20*%20Smith

Signature basestring:
GET&https%3A%2F%2Fapi.twitter.com%2F1.1%2Fusers%2Fsearch.json&count%3D20%26oauth_consumer_key%3DmbmuCGVFTGHZOo5zr5Sx5A%26oauth_nonce%3DaAKXW%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1357141536%26oauth_token%3D819797-L587qRErTHNXXr8kb0eSqj6uCO2xephn1c14RCfies%26oauth_version%3D1.0%26q%3DBob%2520%252A%2520Smith

Authorization header:
OAuth oauth_version=“1.0”, oauth_nonce=“aAKXW”, oauth_timestamp=“1357141536”, oauth_signature_method=“HMAC-SHA1”, oauth_consumer_key=“mbmuCGVFTGHZOo5zr5Sx5A”, oauth_token=“819797-L587qRErTHNXXr8kb0eSqj6uCO2xephn1c14RCfies”, oauth_signature=“5Y4TwqOQPJQTLtVRhfHWQNhLhdk%3D”

ebshih · 2013-01-02T19:36:28.000Z

Thanks Taylor. I will dig deeper to see if there is unusual behavior happening on my app side, but that is so weird. Thanks for the response - I will see what I find.