Unreliable hashtag search


#1

I have an app which has been successfully searching through the API for posts based on their hashtags for about a couple of years, but which has recently been getting erratic results. For example, up until about Wed/Thu it would return tweets containing #AfrikaEye2018 (a film festival). Since then, it returns nothing. It has been retrieving tweets hashtagged #ClimateChange for months, but recently it does so unreliably. If I do a search from a user account with these hashtags, I largely see what I expect to see, although even that doesn’t produce all the posts that have in fact been made.

Has something changed in the API or terms of service? In case it’s relevant, my query seeks only tweets containing videos. I haven’t yet sought approval for the account from which the API searches take place. As I say, it’s been working for two years or so. The API rate limits have not been reached.

Many thanks for any help.

Tim


#2

Hey @timkindberg,

If you are using the standard search/tweets endpoint, then you should note the following statement on that API reference page:

Please note that Twitter’s search service and, by extension, the Search API is not meant to be an exhaustive source of Tweets. Not all Tweets will be indexed or made available via the search interface.

We have not made any changes to this endpoint in a long time, so it is strange to hear about this change in behavior.

What I can recommend to you is to apply for a developer account and start working with the newer premium Search API which will return a complete set of results, unlike the standard search/tweets endpoint.


#3

Thanks for your reply. I am using the search/tweets endpoint. I’m aware of the terms of service that you cite. It’s striking, however, that the results have suddenly become very unreliable, whereas I’ve found them to be reliable in practice for retrieving recent tweets for the last two years. E.g. literally nothing from #AfrikaEye2018 is being returned as of a few days ago; and the results for #ClimateChange tend to be a few consecutive tweets, then a larger gap, then a few consecutive tweets, etc.

Has anyone else noticed this behaviour?

In any case, I’ll apply for a developer account


#4

Just done a test of this myself with the tag #AfrikaEye2018 it returns results from today back to the 6th Nov. Are you posting the recent attribute with the query ?