Search API returning less result than usual

[PruebasMolonas]

Yes, thanks Andy.

[camonmon]

Hi @andypiper any news about this. It seems to go better, but still many tweets missing. Maybe something to do with the #SB50? Thanks

[andypiper]

There is no correlation here with #SB50.

As I understand it, the primary issue here was caused by a temporary problem with a batch of Tweets being accidentally labelled as spam and excluded from the index, as I mentioned above. We fixed that issue and corrected that batch so that they were indexed correctly.

I’m not aware of any other search quality issue at the moment, so I’d need more detail of what you’re seeing. Remember that the public search API is not a complete index of all accounts and Tweets, is optimised for recency, and only covers a limited timeframe.

[impetusashwin]

Hi, I am using the Search API for finding most recent tweets. I am not receiving the “max_id” for getting the next results after few requests (sometimes after 17 requests and sometimes after 45 requests). I am using app-only authentication and the rate limit is also not reached (450 req / 15min window).

Can someone tell if i am missing something in the API request. I am trying to retrieve most recent tweets for past 7 days using max_id. Thanks!

[andypiper]

FYI to complete the loop here, I’ve just seen another small change committed that should hopefully have closed out any lingering issues related to this fault.

[PruebasMolonas]

Hi Andy

This is giving problems again. Other topics have been opened in the forum. The SEARCH API is giving problems, returning less tweets.

[andypiper]

The other topic you commented on was from a few weeks ago. If there is an issue today, please can you provide some additional specifics (terms, types of content you’re searching for, etc) in order for me to check in with the team? Thanks.

[PruebasMolonas]

Hi Andy, thanks!

For example the hashtag #HUBDAY. I tried all different tools: hashtracking, Tweet Binder, tweetreach and so and all of them return only 50 to 60 tweets from the last hours when the hashtag has lot more.

[andypiper]

I had confirmation from the team that there were a few issues with that term that should be settling out.

[PruebasMolonas]

Thanks Andy. We will stay tuned. Thanks @andypiper

[PruebasMolonas]

Hi @andypiper any news about this? We see that the API is returning less results in many hashtags. Would you need more info about those hashtags? Thanks

[andypiper]

The HUBDAY that you reported the other day should have cleared. There was an issue with an indexing job.

If you are reporting search quality issues we need to understand the specifics (hashtags, terms, expectation / reason or evidence for the report) please. That being said, we also can’t investigate every instance of a search issue, as that could get incredibly time-intensive. Happy to look further if proven necessary.

[PruebasMolonas]

Hi @andypiper, first of all, let me tell you in behalf of the coders here that you do a great job by listening to users.

Second, I may have a hint. It happens that the hashtags that give bad results (less than expected) are cut at one time. For example: #eShowBCN16 the hashtag only returns results since 15:56 UK time. It doesn’t matter when you carry out the search, it always starts at 15:56 UK time (16th of March).

Or for example #PedernalesConDanilo they always cut cut, however if I perform the search #PedernalesConDanilo -RT (to avoid RTs) I get many more tweets. Twitter API gives 100% of the results after one date. I have many examples if your team needs something to investigate.

Thanks again Andy

[andypiper]

Thanks, this is helpful detail.

I have not been able to look yet as it is late and I’m away from the office, but sometimes this might happen if, for instance, users flag content with that hashtag as spammy or abusive or low quality, causing our index filters to lower their priority on the API.

I’ll try to learn more on these individually tomorrow but I cannot promise to suddenly “boost” volume results.

[rkueny]

Hi,

I’v the same issue with : https://api.twitter.com/1.1/search/tweets.json?q=%23jecroisenmoi no results return but I’ve results on offical search : https://twitter.com/search?f=tweets&vertical=default&q=%23jecroisenmoi&src=typd

Any solutions ?

[PruebasMolonas]

Hi @andypiper same problem with #sligohour it returns only 23 tweets in the last hours when it has 3000 tweets in the last day.

[PruebasMolonas]

More hashtags returning less results #code307 and #CIUGENIN for example. There are thousands of tweets but the API only returns a couple dozens.

[PruebasMolonas]

Hi again @andypiper this problem continues and it’s getting a real issue. When looking for results on certain hashtags, the API returns only a few tweets when there are thousands, for example:

• #BrightonSEO
• #SEOcafe

Twitter’s API detects there is SPAM and then stop returning results. @andypiper, do you need anything else from us? More info?

Thanks man

[andypiper]

I really cannot help at the moment as I have other priorities. I can report this to the search quality team but there’s no specific steps that can enable the public search API to return Tweets where the antispam tools have blacklisted terms of Tweets.

For full fidelity search you should be looking at the Gnip products.

[TweetinviApi]

Have you tried using the FilteredStream while they are fixing the problem?