Search API Not Returning Recent Tweets

markzohar · 2015-12-08T04:58:09.280Z

We began experiencing issues with the Search API not returning recent tweets this afternoon. When passing a query to the Search API, the API is not returning any tweets more recent than around 5:30pm ET today (Dec 7). The API status page indicates that there are no performance or quality issues with the Search API but we are consistently not receiving recent tweets for any query. However, when testing the same queries on the Twitter website, the search results do include the most recent tweets.

Can you please confirm whether there are current issues with the Search API that are preventing the most recent tweets from being returned. Any help or guidance would be appreciated.

Thanks.

rajeevedmonds · 2015-12-08T06:25:11.078Z

I’m not getting correct results since morning. I’ve tested it with Tweetdeck (web) and the native web interface too.

My problem is not connected with recent tweets. Query is just pulling incorrect results.

Query is → keyword OR keyword OR “phrase 1” OR “phrase 2” lang:en -filter:links ?

It has been working correctly ever since. This problem has started since today morning (IST).

markzohar · 2015-12-08T12:10:38.135Z

It appears that there is a bug with the Search API that affects queries that filter on links. It looks like the indexer broke around 22:00 GMT yesterday and is not recording the fact that any tweets have links in the index. Queries that don’t filter on links are returning recent tweets but queries that filter on links are not returning any tweets after approx 22:00 GMT Monday, Dec 7th. cc: @andypiper

andypiper · 2015-12-08T16:05:56.668Z

Thanks - we are looking into this.

brianpetro · 2015-12-09T11:46:42.488Z

Should we change our systems to avoid this filter? Or will this be permanently fixed, soon?

andypiper · 2015-12-09T12:03:23.702Z

Still under investigation, so if this is impacting you I’d suggest dropping that filter for the moment, until we can provide more guidance.

brianpetro · 2015-12-09T12:15:33.132Z

Thanks @andypiper

loicginoux · 2015-12-11T12:58:42.566Z

Hi @andypiper , do you have more information on this? It seems we have the same issue here and would like to know if we need to display an appropriate message accordingly for our users or if this is something that’s going to be resolved in the next few days.

Thank you.

andypiper · 2015-12-11T14:21:16.684Z

No news at the moment, being tracked internally as PREL-17345.

markzohar · 2015-12-11T14:40:13.898Z

Hope this doesn’t end up like the geo bug from last year. Was tracked internally for a year only to be informed last month that it will never be fixed. Say it ain’t so, Andy!

andypiper · 2015-12-11T20:21:45.640Z

Yes I understand that concern. Will pursue as best we can. Stand by for updates.

andypiper · 2015-12-17T11:48:56.438Z

We’re not currently able to reproduce this.

If you’re able to share a specific search which is failing / exhibiting this issue, it would help us tremendously. Specifically, if you can share the specific request URL with the results you get / the results you expect, that would be useful. Additionally, if you can demonstrate using the API console https://dev.twitter.com/rest/tools/console we can rule out some other variables that might affect our ability to reproduce.

Thank you for your patience, this is being looked into.

reynaldborer · 2015-12-17T12:39:39.833Z

Hi,

I’ve tried the same search that was failing 6 days ago using rest console (https://api.twitter.com/1.1/search/tweets.json?q=%23StarWars&result_type=recent&count=10&since_id=677466830705958911&include_entities=true) and it also works fine now ?!?

lphilps · 2015-12-17T14:15:45.894Z

Last week, queries like this,

https://api.twitter.com/1.1/search/tweets.json?q=Technology+filter%3Alinks&count=100&result_type=recent&lang=en

that contained the filter:links parameter, suceeded, but didn’t return any tweets newer than 2015-12-07 22:30 GMT even long after that time had passed. However, if I removed the filter:links parameter and thus ran this query

https://api.twitter.com/1.1/search/tweets.json?q=Technology&count=100&result_type=recent&lang=en

it suceeded also, but returned tweets up to the current time.

I just retested this and the bug appears to have gone away. That first query above is now returning very recent tweets. So, that’s good.

That being said, just FYI, even with the flag present, in my testing, about 5% of the tweets it returns don’t contain a link, but … that’s not a big deal.

I’ll confess however that It’s a little worrisome that the problem seems to have gone away all on it’s own.

andypiper · 2015-12-18T17:09:42.644Z

We have a working hypothesis on what might have happened on our side (involving traffic routing) given we couldn’t reproduce some days later. Apologies for the short disruption, though!