No accurate information on the IPs used by the Twitterbot Crawler


#1

Hi there!

  • Twitter uses the Twitterbot crawler to fetch content for cards and so forth
  • Twitter does not seem to publish information that allows website owners to verify the identity of users claiming to be the Twitterbot
  • The information that website owners need is either:
    • a list/range of IP addresses associated with legitimate requests coming from this bot
    • OR a common set of domain name(s) that IP addresses relating to this bot will resolve to when doing a reverse lookup on the IP address

It would be great if Twitter could update their docs re: this so that we could get accurate information that would allow us to verify the identity of requests coming from a user claiming to be the Twitterbot :slight_smile:

Many thanks!


What is the twitter robot IP address I should whitelist?
#2

The documentation states:

Twitter’s aggregate outbound IP ranges are 199.16.156.0/22 and 199.59.148.0/22. Twitter’s ASNUM is AS13414.

I’m certainly not aware that we would have switched to 199.59.150.* and that does not appear in the ASNUM listing for IPv4. I’ll definitely take this up internally to check if anything has indeed changed.