Sudden increase in 420 (Exceeded connection limit for user) responses from Streaming API


#1

Hi, since around 17:30GMT on 14/4/2014 we’ve seen a sudden increase in the number of 420 responses we’re getting form the Streaming API. ie. from zero, to happening once every 5-10 connections or more.

We’ve seen this happening across a couple of different different data centres around the world, so it doesn’t seem to be geographically specific. The client in question uses track, not follow, and tracks around 300-400 terms simultaneously. It gets restarted every 5 minutes to change the terms it’s tracking. We’ve been using this system continuously for a number of years without any problems.

The error we’re getting - ‘Exceeded connection limit for user’ - seems to indicate that the Twitter systems think we have more than one connection open for the credentials we’re using, however we don’t.

We’ve tried several things so far here to try to figure out what might be going on.

  • Created and tested using a fresh account and API keys to ensure the issue isn’t connected to a specific account problem.
  • Verified that our client properly implements an exponential backoff when a 420 is received, starting at 60s.
  • Added a ten second pause between disconnect and reconnect to mitigate any issues with the servers believing we’re still connected.
  • Increased the reconnect interval of our client from 5 to 10 minutes (this decreased the 420 rate as we were reconnecting less often, but didn’t resolve the problem).
  • Updated our client library (twython) to the latest available version to try to ensure there are no errors in client connection logic.

Could you please advise whether there has been some change to the way client connection limits are being applied which might be causing this problem. Or if there is some other ongoing issue with 420 responses.

Regards,
Alex Kelly
Lead Developer
Smesh (http://sme.sh/)


Increase in 420:Enhance your calm
#2

Alex,

Would you please DM me on Twitter with your email address?

I’d like to get some additional diagnostic information from you when this happens. In particular, I’d like to see the raw HTTP request and response, especially the X-Connection-Hash response header.

Thanks!


#3

Hi Arthur. Thanks for getting back to me - can’t DM you as you’re not following me. If you could do that I’ll send over my email and some more info.

I’ve added some logging of request and response headers to the client and now have some logs from from 200 & 420 responses I can send you.

Thanks,
Alex


#4

I am seeing the same issue. 5 or 6 of these 420 errors starting on 4/14/2014 and continuing through today. Nothing else has changed on my end, and the net total of 420 errors in the past was 2 in about ~6 months time.

I would love to know how this issue can be resolved.


#5

Yes I have the same issue.Please advise.


#6

Hi David & Bill. I’m doing various diagnostics here to try to get to the bottom of this problem.

Could you possibly let me know what clients you are using to connect to the stream? We’re using Twython here, and it would be good to know if the problem is happening on a variety of different clients without having to test them all myself :slight_smile:

Thanks,
Alex


#7

I am using the R package streamR. I get the same errors in Python. I guarantee you this error is not coming from our end–it is timed perfectly with the “upgrade” that happened on April 14th. I have created an entirely new account to see if it had something to do with rate limits, but no, it happens on the new account irrespective of connection history.

I am extremely annoyed that no fix/issue has been made of this yet, since it’s definitely not just us experiencing this problem.


#8

Getting the same problem using Twitter4j. It almost seems like when I restart my stream my old one is not getting disconnected?


#9

Same issue here. A lot of the users of my Android app are seeing their timeline stuck when using streaming.
A did a few tests on my devices and it seems the rate limit is never removed, even if I have only one connection active and I wait for a long period of time before reconnecting.


#10

Wow, I was just looking into this today because I noticed it on my android app as well, Talon. Same deal, started around a week ago or so and it is constantly trying to reconnect and giving me the 420 error code. The stream never disconnects even when I try to do it manually. It just keeps trying to establish a connection until I restart the device.

Guess I am glad that I am not alone here, hopefully we can figure out some way to fix it!


#11

Unfortunately Twitter doesn’t seem to really care about this issue.
The userstream endpoint has been reported as “disrupted” for the past 5 days and they haven’t said a word about that.


#12

I’m having similar problems. Many users have reported problems with the streaming behavior of my client (Tweetium), and I’ve also been encountering far more problems with 420 errors on my push notification server.

I’m curious if these are the same problem or two different ones. I keep hoping the ongoing (and seemingly acknowledged) service disruption will be addressed and then we can see if the increased 420 response problem is also fixed.


#13

Regenerated my keys and everything seemed to work again.


#14

@julioadl I’m not sure you’re having the same issue as some of us–I created entirely new test accounts and still see the 420 issue. In our case it’s not due to connection or rate-limiting, but rather random failures to connect when interacting with the streaming API on a predefined interval.


#15

Still not a single word from Twitter…nice…


#16

We are investigating the issue, it’s not clear to us at this time what is the root cause of the disconnection. It might be related to https://dev.twitter.com/discussions/27003 but not 100% certain.

Tracking the issue in https://dev.twitter.com/issues/1648.


#17

We have found the root cause and will be deploying a fix tomorrow during the day.


#18

Thank you! That’s good to hear.


#19

Thanks for the update Sylvain. We’ve seen the 420 responses disappear since earlier this morning, so I guess you’ve made a change already and it seems to have fixed the problem.


#20

The deploy this morning should fix this issue for everyone. Please let us know if you still have this problem with errors 420.