Url-encoded characters cause "Missing or invalid url parameter" in search/tweets


#1

Some queries in search/tweets.json are returning the error "Missing or invalid url parameter" for seemingly random reasons, when using the url: selector, with a value containing url-encoded characters.

Example:

$ twurl /1.1/search/tweets.json?q=%22url%3A%22http%3A%2F%2Fthaitechnewsfeed.com%2F2017%2F06%2F02playground-%25e0%25b9%2580%25e0%25b8%2595%25e0%25b8%25a3%25e0%25b8%25b5%25e0%25b8%25a2%25e0%25b8%25a1%25e0%25b8%25ad%25e0%25b8%25ad%25e0%25b8%2581%25e0%25b9%2580%25e0%25b8%25a7%25e0%25b8%25ad%25e0%25b8%25a3%25e0%25b9%258c%25e0%25b8%258a%25e0%25b8%25b1%25e0%25b9%2588%25e0%25b8%2599%25e0%25b9%2583%25e0%25b8%25ab%2F%22

(q = url:"http://thaitechnewsfeed.com/2017/06/02playground-%e0%b9%80%e0%b8%95%e0%b8%a3%e0%b8%b5%e0%b8%a2%e0%b8%a1%e0%b8%ad%e0%b8%ad%e0%b8%81%e0%b9%80%e0%b8%a7%e0%b8%ad%e0%b8%a3%e0%b9%8c%e0%b8%8a%e0%b8%b1%e0%b9%88%e0%b8%99%e0%b9%83%e0%b8%ab/" )

Returns:

{"errors":[{"code":195,"message":"Missing or invalid url parameter."}]}

Here is a shorter version that also returns an error:

twurl /1.1/search/tweets.json?q=%22url%3A%22%25e0%25b9%2580%25e0%25b8%2595%25e0%25b8%25a3%25e0%25b8%25b5%25e0%25b8%25a2%25e0%25b8%25a1%25e0%25b8%25ad%25e0%25b8%25ad%25e0%25b8%2581%25e0%25b9%2580%25e0%25b8%25a7%25e0%25b8%25ad%25e0%25b8%25a3%25e0%25b9%258c%25e0%25b8%258a%25e0%25b8%25b1%25e0%25b9%22

This is not a length issue, because I’m able to get a proper response with a longer query:

twurl /1.1/search/tweets.json?q=%22url%3A%22http%3A%2F%2Fthaitechnewsfeed.com%2F2017%2F06%2F02playground-%25e0%25b9%2580%25e0%25b8%2595%25e0%25b8%25a3%25e0%25b8%25b5%25e0%25b8%25a2%25e0%25b8%25a1%25e0%25b8%25ad%25e0%25b8%25ad%25e0%25b8%2581%25e0%25b9%2580%25e0%25b8%25a7%25e0%25b8%25ad%25e0%25b8xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx%22
{"statuses":[],"search_metadata":{"completed_in":0.051,"max_id":872728129973870592,"max_id_str":"872728129973870592","query":"%2522url%253A%2522http%253A%252F%252Fthaitechnewsfeed.com%252F2017%252F06%252F02playground-%2525e0%2525b9%252580%2525e0%2525b8%252595%2525e0%2525b8%2525a3%2525e0%2525b8%2525b5%2525e0%2525b8%2525a2%2525e0%2525b8%2525a1%2525e0%2525b8%2525ad%2525e0%2525b8%2525ad%2525e0%2525b8%252581%2525e0%2525b9%252580%2525e0%2525b8%2525a7%2525e0%2525b8%2525ad%2525e0%2525b8xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx%2522","refresh_url":"?since_id=872728129973870592&q=%2522url%253A%2522http%253A%252F%252Fthaitechnewsfeed.com%252F2017%252F06%252F02playground-%2525e0%2525b9%252580%2525e0%2525b8%252595%2525e0%2525b8%2525a3%2525e0%2525b8%2525b5%2525e0%2525b8%2525a2%2525e0%2525b8%2525a1%2525e0%2525b8%2525ad%2525e0%2525b8%2525ad%2525e0%2525b8%252581%2525e0%2525b9%252580%2525e0%2525b8%2525a7%2525e0%2525b8%2525ad%2525e0%2525b8xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx%2522&include_entities=1","count":15,"since_id":0,"since_id_str":"0"}}

Here is the minimal query that seems to produce the error:

twurl /1.1/search/tweets.json?q=%22url%3A%22h%3A%2F%2Ft.co%2F2%2F0%2F0-%25e0%25b9%2580%25e0%25b8%2595%25e0%25b8%25a3%25e0%25b8%25b5%25e0%25b8%25a2%25e0%25b8%25a1%25e0%25b8%25ad%25e0%25b8%25ad%25e0%25b8%2581%25e0%25b9%2580%25e0%25b8%25a7%25e0%25b8%25ad%25e0%25b8%25a3%25e0%25b9%22

Removing any character in this query makes the error disappear.

I’m seeing this error a lot, when the searched URL contains urlencoded asian or arabic glyphs.


#2

Updated the examples because I’m not sure what the query meant, when translated.