Connections to API timing out


#1

EDIT2:
The issue was on our side. A line card dropped TCP traffic intermittently without generating alarms.


I’m seeing connections to https://api.twitter.com time out since the 2nd of April.

  • connect to 104.244.42.130 port 443 failed: Connection timed out

I’m getting OK responses most of the time that include the header:

server: tsa_b

EDIT
This is an example of a full request that times out at the first try and works on the 2nd attempt that curl takes:

curl -v --user-agent "whytimeout" https://api.twitter.com/index.json
*   Trying 104.244.42.66...
* TCP_NODELAY set
* connect to 104.244.42.66 port 443 failed: Connection timed out
*   Trying 104.244.42.194...
* TCP_NODELAY set
* Connected to api.twitter.com (104.244.42.194) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /opt/nginx/embedded/ssl/certs/cacert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=San Francisco; O=Twitter, Inc.; OU=Twitter Security; CN=api.twitter.com
*  start date: Jun 29 00:00:00 2016 GMT
*  expire date: Sep 19 12:00:00 2019 GMT
*  subjectAltName: host "api.twitter.com" matched cert's "api.twitter.com"
*  issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert SHA2 High Assurance Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x1d41d00)
> GET /index.json HTTP/2
> Host: api.twitter.com
> User-Agent: whytimeout
> Accept: */*
> 
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 404 
< content-length: 68
< content-type: application/json;charset=utf-8
< date: Wed, 04 Apr 2018 22:04:17 GMT
< server: tsa_b
< set-cookie: personalization_id="v1_dTBcEhqIA48FOcxbMTdtyQ=="; Expires=Fri, 03 Apr 2020 22:04:17 UTC; Path=/; Domain=.twitter.com
< set-cookie: guest_id=v1%3A152287945735007275; Expires=Fri, 03 Apr 2020 22:04:17 UTC; Path=/; Domain=.twitter.com
< strict-transport-security: max-age=631138519
< x-connection-hash: ec2d83886c7dc7b78de42f97a282de74
< x-response-time: 5
< 
* Connection #0 to host api.twitter.com left intact
{"errors":[{"message":"Sorry, that page does not exist","code":34}]}

MTR doesn’t show any lost packets.

I’m able to reproduce this from multiple hosts on the same network.

Thank you,
Florian


#2