API Timeouts from a NodeJS app

[Zappatta]

Hello,

We are currently experiencing highly frequent timeouts from some of our machines. The symptoms are rather identical to what is described in issue 100354. I.E. our NodeJS app emits ETIMEDOUT when trying to connect to the api.twitter.com IP addresses.

When doing curl https://api.twitter.com from the same machines that fail, we are seeing a 100% success rate in receiving response. So this might be a problem with some specific endpoints only, or with NodeJS apps only.

Unfortunately, I cannot provide ping/traceroute/mtr logs due to ICMP limitations on the network we cannot circumvent at the time being.

TCP dumps show that a connection is actually established with the twitter API endpoint, and that data is being transmitted. But since it is encrypted I cannot tell what the data contains. The session is finally terminated with FIN, ACK packages which are then followed by RST packages.

Since curl works, and since we are able to flawlessly connect from other machines on other networks, our immediate suspicion is that the problem is not on Twitter’s end. Nonetheless, I am posting here to see if anybody else encountered similar problems and had any luck resolving them. Also, if the Twitter staff can assist us with debugging (such as checking what happens to the sessions on your end) this it would be tremendous help!

Many thanks!

[LeBraat]

I’m assuming you are talking about the statuses/filter endpoint here, but please correct me if I’m wrong.

[Zappatta]

Hey Lebraat. Thank you for your reply!
I am Indeed talking about the statuses/filter endpoint.

[andypiper]

I’m not aware of a way for us to check for the connections and sessions on our side.

One question since you’re connecting to the streaming statuses/filter endpoint - this endpoint requires an ongoing open connection (not a request/response), so you’ll need your code to be capable of holding an open connection. Also be aware of the limitation that you cannot have more than a couple of connections to the streaming API from a single app ID, and you may not use multiple app IDs for the same use case in order to circumvent this restriction. Do you have existing connections from the same IP and/or app ID?

[Zappatta]

Thank you andypiper for your reply. I must apologise sincerely, but after further examination it appears that I gave you the wrong information . These errors appear to be originating from the /search/tweets endpoint.

Once again, apologies for my error. And thank you kindly for your time and help.

[andypiper]

So to clarify / summarise:

• most machines in your network have no issue hitting api.twitter.com endpoint from code and curl
• some machines sometimes / frequently see a ETIMEDOUT error hitting the search/tweets endpoint from node
• it appears that the code successfully connects and receives data but then times out and the connection closes

As I mentioned, unfortunately there’s no way for my team to look into individual connection attempts. Have you tried adding additional tracing information (such as looking into TCP request / response headers)?

[bobber205]

I’ve run into this before and I think it’s a NodeJS issue – something about certain HTTPS connections are hard for a Node app to remain connected to.

I used to have a link that explained this but I have misplaced it.

[Zappatta]

That is a pretty much accurate summarisation, yes. But i’ll also stress out that the failing machines are running on a different architecture (Kubernetes, different VLANs).

Unfortunately, since the error seems to occur on a TCP level, the error traces does not provide information about what is happening at application level.

We haven’t closely examined the request/response HTTP headers, but if you suspect it may provide more information we can easily log them to take a closer look. We’ll do so, and post our updates. If you have any more ideas where we can look, we’d be delighted to hear them this is a needle in haystack for us right now.

Thanks once more for the help

[Zappatta]

Thanks for this bit of information. We do suspect this has something to do with NodeJS but cannot pinpoint it. if you ever do find that link, please do share

[system]

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.