Search API next_url "q" parameter encoding error

[bmaso]

There seems to be a new bug in the Search API. The “next_url” is not correctly encoding escaped characters.

Example: when you do a search with the query parameter ‘…q=%23bacon…’, which is a search for the hashtag “#bacon”, the first page of the response comes back correctly, but with an incorrect “next_url” response value. What you get back in the “next_url” includes the query parameter
’…q=%2523bacon…’. That is, the “%” character is being encoded, so that “%23” in the original query is becoming “%2523” in the “next_url” value.

A workaround is to explicitly change “q=%2523” in to “q=%23” – which is a big kludge, but gets the job done.

Brian Maso

[episod]

Thanks for the report Brian.

I can’t reproduce at the moment – can you provide some kind of trace on the exact query you initially send in and the raw response you’re getting back from the API?

I send my initial request:

GET /1.1/search/tweets.json?q=%23bacon HTTP/1.1


This is the metadata I get back, with #bacon encoded a single time as I would expect:

"search_metadata":{"completed_in":0.046,"max_id":360774524679102464,"max_id_str":"360774524679102464","next_results":"?max_id=360772806599913471&q=%23bacon&include_entities=1","query":"%23bacon","refresh_url":"?since_id=360774524679102464&q=%23bacon&include_entities=1","count":15,"since_id":0,"since_id_str":"0"}}

If I then walk to the next page it gave me, I still get a valid looking metadata:

“search_metadata”: {
“max_id”: 360772806599913471,
“since_id”: 0,
“refresh_url”: “?since_id=360772806599913471&q=%23bacon&include_entities=1”,
“next_results”: “?max_id=360769982516301823&q=%23bacon&include_entities=1”,
“count”: 15,
“completed_in”: 0.03,
“since_id_str”: “0”,
“query”: “%23bacon”,
“max_id_str”: “360772806599913471”
}
}


If you’re able to reproduce this in a curl request and can include the HTTP headers you got back in the response, I can look into whether there’s some kind of rogue server in the pool or something causing this for you.

[bmaso]

(* sheepish *) I think maybe it was a bug in my proxy, which seems to be incorrectly translating the response JSON… Sorry for the misinformation…

Brian Maso

[episod]

No problem! It happens I’m definitely guilty of filing too many bugs against the engineering teams around here that were just a result of my contrived environment!