Twitter search for domain name fails

[ArturBrugeman]

Hello,

Recent days I see a problem with domain name searches on Twitter.

For some domain names Twitter returns no results:
https://twitter.com/search?f=tweets&q=growth-hackers.net&src=typd

However, when I search for specific post, some tweets are returned:
https://twitter.com/search?f=tweets&vertical=default&q=http%3A%2F%2Fwww.growth-hackers.net%2Fgrow-startup-revenue-blog%2F&src=typd

This happens occasionally, and for other domain names things work fine. But I’ve seen this issue several times the last week and decided to raise this topic.

Does anyone have an explanation for this?

Thank you.

[andypiper]

Is this an issue with the API, or with search on the website? We can’t help with issues related to twitter.com here.

There are a variety of reasons why results may not appear in the API - the index is not complete, as the documentation states, and you only have access to 7 days of history.

[ArturBrugeman]

Hi Andy, thank you for your response.

The issue relates to both Twitter.com and the REST API. And yes, I know the index is incomplete and there have never been any guarantees.

However:

1. I have not seen such inconsistencies before (related to domain name search), so this seems to be something new.
2. To me, ‘incomplete’ index would mean that some tweets could be indexed while others could be missing. In this case no tweets ever appear for specific domain name search, not some select tweets. This is just strange.

So, I’m asking because this feels new and strange, and thought this might be something worth looking into.

Thank you!

[andypiper]

OK thanks for the additional context.

FWIW I checked in the Gnip search API (which comprises the complete index), and I see Tweets using these URLs.

There’s a chance that the URLs are in Tweets that have been marked by some users as spammy, and thus are not showing up in the public search results on Twitter.com as a safety measure. This is not a new behaviour, but it may be new in the case of these particular links. That would be my theory in this case, but I can’t be certain.

[ArturBrugeman]

Thank you, Andy.

My first message contains two search queries, one (for domain name) returns nothing, the other (for urls from same domain) returns several recent tweets. If tweets were marked as spam, we wouldn’t see them in both queries (my guess).

Please, if there is a chance you guys could look a little deeper?

We rely on domain name search to track shares, and recent examples of described behavior breaks this functionality completely.

Thank you very much!

[andypiper]

What I see when I retrieve Tweets from the Gnip index is that many of the Tweets are using a variety of link shorteners so it could be related to that. I’ll see if I can find anything out, but it may not be easy to identify what is happening here.

[ArturBrugeman]

Thanks a lot, Andy! I know shorteners, canonicals and crawling errors all make life harder, so thank you very much for trying to help! Hope to hear from you soon.

[ArturBrugeman]

Hi Andy, is there any updates on this issue?

Thank you.

[andypiper]

Very sorry but I don’t have any additional information to share at this time. Is this continuing to be an issue?