What is the twitter robot IP address I should whitelist?

whitelisting
robots

#1

Hi,
What is the twitter robot IP address I should whitelist?

Currently I have below 4 IP addresses crawl my web site very aggressively and sometimes make my web site very slow:

199.16.157.180
199.16.157.181
199.16.157.182
199.16.157.183

I don’t know whether I should block it or not.

There is a post saying that the twitter bot ip address can be found in the twitter troubleshooting guide,

but I can not find the place indicating the IP address I should whitelist in the troubleshooting guide, can anyone point out the URL?

Thanks
-David


#2

I too would love to know if there is a list of IP addresses associated with the Twitterbot published somewhere?

All other major bots let you do reverse DNS lookups to check the authenticity, but I can’t seem to find this anywhere for the Twitterbot. All the information I have found seems to be inaccurate.

Can a twitter dev give us some more insight? :slight_smile:


#3

Yes this information is documented.

Please see this page.
https://developer.twitter.com/en/docs/tweets/optimize-with-cards/guides/troubleshooting-cards


#4

That information appears to be inaccurate - we have had legitimate requests that have come from the Twitterbot outside of the 199.59.156.* IP address range specified on that page.

I’ve created another more detailed topic here: No accurate information on the IPs used by the Twitterbot Crawler