No accurate information on the IPs used by the Twitterbot Crawler

[tyronedougherty]

Hi there!

• Twitter uses the Twitterbot crawler to fetch content for cards and so forth
• Twitter does not seem to publish information that allows website owners to verify the identity of users claiming to be the Twitterbot
  • The only official information I can find on this is located here: https://developer.twitter.com/en/docs/tweets/optimize-with-cards/guides/troubleshooting-cards
  • It seems suggests that the range is 199.59.156.*, but we have found this to be inaccurate as requests seem to come from the 199.59.150.* range instead?
• The information that website owners need is either:
  • a list/range of IP addresses associated with legitimate requests coming from this bot
  • OR a common set of domain name(s) that IP addresses relating to this bot will resolve to when doing a reverse lookup on the IP address
    • This is preferable because IP addresses change, and it’s a much more accurate way of performing bot verification
    • A good example of how Google does this is here: https://support.google.com/webmasters/answer/80553

It would be great if Twitter could update their docs re: this so that we could get accurate information that would allow us to verify the identity of requests coming from a user claiming to be the Twitterbot

Many thanks!

[andypiper]

The documentation states:

Twitter’s aggregate outbound IP ranges are 199.16.156.0/22 and 199.59.148.0/22. Twitter’s ASNUM is AS13414.

I’m certainly not aware that we would have switched to 199.59.150.* and that does not appear in the ASNUM listing for IPv4. I’ll definitely take this up internally to check if anything has indeed changed.