Reason for getting blocked and how to avoid that


#1

Hello Twitter,

we are using your Rest API v1.1 and have recently been blocked two times based on our IP address (212.101.212.71). In consequence we were not able to reach api.twitter.com anymore. As far as we could see not just the mentioned IP had been blocked but several Class C networks that we and others use. In both cases we stopped using the API from the mentioned IP and bacame unblocked about a day later.

Is there a way to get information about the reason for blocking our IP? Also what can we do to avoid getting blocked again?
We would like to set up a monitoring to make sure, that we can reach api.twitter.com. Is this allowed and what do we have to pay attention on? Of course we don’t want to get blocked because of our monitoring.

Thanks a lot and best regards
Julian


#2

First a good thing to check is that your server is up to date, time is in sync, and OAuth is working properly - most of the time when it seems like “your IP is blocked” it’s actually an expired token, your clock is out of sync (OAuth doesn’t work) or your libraries are out of date and trying to make a request to a deprecated endpoint. Use the CURL test tool on one of the REST endpoints to generate a valid request.

Another thing to keep in mind are rate limits - assume that any endpoint that doesn’t have a ratelimit listed in the chart https://dev.twitter.com/rest/public/rate-limits is 15 calls per 15 minutes. Twitter responds with ratelimit info in response headers - you can use those to keep an eye on requests and stop when you run out of calls. It’s a very bad idea to call the same endpoint from multiple threads - your rate limits will be unpredictable, make 1 call at a time per endpoint instead.

Monitoring to make sure api.twitter.com is reachable is a bad idea: See F.5.d https://dev.twitter.com/overview/terms/agreement-and-policy#6.Update_Be_a_Good_Partner_to_Twitter it’s better to just keep your app running normally, and monitor for error responses.

Log and analyse errors from the API properly, and respond accordingly: see https://dev.twitter.com/overview/api/response-codes eg: if it’s a network timeout, it’s ok to retry after a short wait time, if it’s a rate limit notice - examine the response headers for the appropriate rate limit reset time, add on a few extra seconds and do not make any more requests to the endpoint. If it’s a 401, 403, 404 type error: do not make the request again. If it’s a problem with the request (invalid parameters, too many characters or whatever) fix it before retrying.

You’re unlikely to be blocked for making a few errors occasionally, or even running the maximum allowed calls frequently - but you are very likely to be blocked if you ignore errors and just keep retrying calls that are failing.


#3

Thanks for your comprehensive answer. We do our best to respect rate limits and we are pretty sure, that we did not exceeded the rate limits in the mentioned cases.

Are you able to answer the following questions?
First of all, is there a way to check the concrete cause for our IP-blockings now afterwards?
If not, is there a way to chech the cause during a IP-blocking if it should occur again?

As we could found out in our cases, several networks had been blocked instead of seperate IPs. Is this normal abuse handling?

Thanks for help in advance.


#4

I guess you can try the same process for blocked IPs as for suspended applications: https://support.twitter.com/articles/72585

I imagine if whole ranges of IPs are blocked - it must be for something substantial, so i’d make really sure it’s not some other error or network configuration issue.