Searching the 30 Day API with the URL operator

[ryanwi]

Could somebody review these and let me know the proper way to use the url operator?

GET request

twurl "/1.1/tweets/search/30day/[my_env].json?query=url:https://github.com/sferik/twitter"

{"error":{"message":"There were errors processing your request: no viable alternative at character '/' (at position 11), no viable alternative at input ':' (at position 10), no viable alternative at character '/' (at position 12)","sent":"2017-11-20T05:52:44+00:00","transactionId":"00e64f6f00dfb114"}}

GET request with escaped query (there are results for this in the search on twitter.com, so this feels like the wrong response.

twurl "/1.1/tweets/search/30day/[my_env].json?query=url%3Ahttps%3A%2F%2Fgithub.com%2Fsferik%2Ftwitter"

{"results":[],"requestParameters":{"maxResults":100,"fromDate":"201710210000","toDate":"201711200556"}}

POST request with serialized JSON

twurl -A "Content-Type: application/json" -X POST -d "{\"query\":\"url:https://github.com/sferik/twitter\"}" /1.1/tweets/search/30day/[my_env].json

{"error":{"message":"There were errors processing your request: no viable alternative at character '/' (at position 11), no viable alternative at input ':' (at position 10), no viable alternative at character '/' (at position 12)","sent":"2017-11-20T06:22:42+00:00","transactionId":"0082557000a003a2"}}

[andypiper]

twurl does not currently support the application-only authentication type that the premium 30-day search uses, so I’d recommend an alternative such as curl or httpie.

The issue here is that you should omit the protocol from the url: query parameter.

For example:

http 'https://api.twitter.com/1.1/tweets/search/30day/development.json?maxResults=10&query=url:github.com/sferik/twitter' 'Authorization:Bearer [mybearertokenlongstring]'

That’s not made explicit in the documentation, so I’ll make a note to get that clarified.

[ryanwi]

Thanks Andy, confirmed that I can get results without the protocol. Updating the docs to make that explicit will be helpful.

FWIW, I was able to use twurl for many other searches, no problems there.

[andypiper]

Good to know, thanks! One bug / issue in twurl worth knowing about is that currently, JSON bodies containing the = character need some massaging to work, just in case you come across that. We’re looking at the best ways to resolve.

[snowman]

Added to the url: description that tokens and phrases with punctuation and special characters should be double quoted. We generally do not recommend trying to match on a specific HTTP/HTTPs protocol, but if you do, this is an example of needing to double-quote it, e.g. url:“https://developer.twitter.com”.

So, for the example above, the rule should be url:“github.com/sferik/twitter”
(escaping these quotes as needed)

[andypiper]

@ryanwi apologies for circling back with you so much later, but I’l just curious to understand which version of twurl you’ve been using to access the 30-day search API. Have you made any patches to it yourself?

[ryanwi]

I’m using 0.9.3 and I have not made any patches myself. As I shared in the other thread, it’s not the most consistent thing, but some searches do work.

$ twurl -v
0.9.3

[lxcichano]

Interestingly, I am, and have been, using the same twurl version (0.9.3), though I’ve yet to get twurl to successfully give me query results for the 30-Day API.