API slowiness problem


#1

I have a server in OVH France.
For last 2-3 days we have big performance issues.
For example generally a GET-/followers/ids takes 700-800 ms.
But last days it takes nearly 4 seconds per request.
This slowiness is similar for our other Twitter REST API requests.

We didn’t change anything in our code and our provider says that no network problem exists.
Also when we check our server with Ookla’s speedtest, it seems good (900 Mb/s download, 200 Mb/s upload)

Does anyone knows what would be the problem for last days?
Thank you.


#2

Also Twitter’s status page says that there exists no performance problems:
https://dev.twitter.com/overview/status

Service / Website	Performance and Availability Status	Current Performance	Uptime Last 24h
/1.1/friends/ids	 Service is operating normally           940 ms	100.0%
/1.1/search/tweets	 Service is operating normally          746 ms	100.0%
/1.1/statuses/home_timeline	Service is operating normally    1846 ms	98.8%
stream.twitter.com	Service is operating normally            595 ms	100.0%
User Streams	         Service is operating normally            546 ms	100.0%

#3

I just want to be sure, is there a limitation for the aplications ?
I mean, does Twitter limits API response speed ?
If so, do they make it by IP or by application id ?

Does this problem originated from Twitter’s rules ?
To check my state where should I write?
https://support.twitter.com/forms/platform page has these options.
what should I select ?

None of the options seems relevant ?


#4

No there is no such limitation on response speed. You can obviously check the regular API request limits.

I would use the “technical question” support option. I’m not aware of any performance issue on the GET followers/ids endpoint - is this the only endpoint you are having an issue with?


#5

@andypiper Thank you very much for response.
We take care of API limits and I’m pretty sure that slowiness is not related with Error 429.
When we get a 429 error, we stop that user’s requests, until rate limit reset timestamp.

We also have performance issues on GET friends/ids, users/lookup, POST friendships/destroy, friendships/create etc.


“technical question” option option recommends user to look for the forum.


#6

Our previous performance: Snapshot link
673 ms average, 402 requests per minute.

Our current performance: Snapshot link
3600 ms average, 350 requests per minute.

(We don’t have CPU, RAM or disk IO problem. CPU lower than 30%, RAM 50%, disk 4%)
(We check server general speed with Ookla’s speedtest. 850 Mb/s download, 200 Mb/s upload)


#7

Two example like this:
1st response took 0.5 seconds,
2nd response took 8.2 seconds.
I put output response info below.

First response (0.5 seconds)

[info] => Array
(
	[url] => https://api.twitter.com/1.1/followers/ids.json?cursor=-1&screen_name=jack
	[content_type] => application/json;charset=utf-8
	[http_code] => 200
	[header_size] => 900
	[request_size] => 624
	[filetime] => -1
	[ssl_verify_result] => 0
	[redirect_count] => 0
	[total_time] => 0.555656
	[namelookup_time] => 2.1E-5
	[connect_time] => 0.124162
	[pretransfer_time] => 0.396316
	[size_upload] => 0
	[size_download] => 6053
	[speed_download] => 10893
	[speed_upload] => 0
	[download_content_length] => 6053
	[upload_content_length] => 0
	[starttransfer_time] => 0.555524
	[redirect_time] => 0
	[certinfo] => Array
		(
		)

	[primary_ip] => 199.16.156.199
	[redirect_url] => 
	[request_header] => GET /1.1/followers/ids.json?cursor=-1&screen_name=jack HTTP/1.1
		User-Agent: tmhOAuth 0.8.4+SSL - //github.com/themattharris/tmhOAuth
		Host: api.twitter.com
		Accept: */*
		Accept-Encoding: deflate, gzip
		Authorization: OAuth oauth_consumer_key="i_hide_this", oauth_nonce="i_hide_this", oauth_signature="%i_hide_this%3D", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1422194200", oauth_token="i_hide_this", oauth_version="1.0"
		X-NewRelic-ID: i_hide_this
		X-NewRelic-Transaction: i_hide_this
		)

Second response: (8.2 seconds)

[info] => Array
(
	[url] => https://api.twitter.com/1.1/followers/ids.json?cursor=-1&screen_name=jack
	[content_type] => application/json;charset=utf-8
	[http_code] => 200
	[header_size] => 901
	[request_size] => 630
	[filetime] => -1
	[ssl_verify_result] => 0
	[redirect_count] => 0
	[total_time] => 8.208005
	[namelookup_time] => 2.8E-5
	[connect_time] => 7.403713
	[pretransfer_time] => 7.795641
	[size_upload] => 0
	[size_download] => 22170
	[speed_download] => 2701
	[speed_upload] => 0
	[download_content_length] => 22170
	[upload_content_length] => 0
	[starttransfer_time] => 7.961289
	[redirect_time] => 0
	[certinfo] => Array
		(
		)

	[primary_ip] => 199.16.156.40
	[redirect_url] => 
	[request_header] => GET /1.1/followers/ids.json?cursor=-1&screen_name=jack HTTP/1.1
		User-Agent: tmhOAuth 0.8.4+SSL - //github.com/themattharris/tmhOAuth
		Host: api.twitter.com
		Accept: */*
		Accept-Encoding: deflate, gzip
		Authorization: OAuth oauth_consumer_key="i_hide_this", oauth_nonce="i_hide_this", oauth_signature="i_hide_this", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1422193103", oauth_token="i_hide_this", oauth_version="1.0"
		X-NewRelic-ID: i_hide_this
		X-NewRelic-Transaction: i_hide_this
		)

#8

OK - will try to look into this tomorrow. As I said I’m not aware of issues that would cause slower responses at the moment, and I’m sorry for the inconvenience.


#9

Same problem for me. Connecting to the API trough OVH too.


Timeout search API since yesterday
#10

Certainly interesting that OVH is a common thing here - I’m aware of another issue which involves OVH as well. I don’t know the OVH folks myself. What does a traceroute look like / how many hops are involved? I wonder if we can do some more diagnostics around that.


#11

Connecting trough OVH and getting the same problems. Ovh says that this is not something they can do something about. So, kind of stuck we are.

It is obviously a ovh problem.


#12

@andypiper I checked our logs and I can say that this slowiness started at 23 January 2015 12:30 pm EET.
Until that day we got 0.1 timeout in 1000 requests. Since that time we have 200 timeouts in 1000 requests.

,I made traceroutes and I pasted result here: http://pastebin.com/jJi0EAaP


#13

Thanks for the traceroute details!

I certainly do not want to say it is “a ovh problem” for sure, as we do not know; but currently I am seeing 4 reports of slowness with devs and apps on OVH infrastructure, so it is something to dig deeper on. Sadly I’m unable to reproduce here.


#15

I hope this will be fixed soon :smile:


#16

I have exactly the same problem from OVH server ( 46.105.102.213 ) and it has been starting 3 or 4 days ago …


#17

Same problem here using OVH


#18

Now talking trough PM with OVH, hopefully they will tell us something soon. I will keep you guys posted.


#19

OVH just told me they are forwarding the issue to the technical team. They hope to have it fix “as soon as possible”.


#21

OVH opened a Ticket regarding their problems with the connection with api.twitter and soon there will be an answer http://status.ovh.es/?do=details&id=8611 let’s hope for a quick solution here. Thanks


#22

When I check our logs, I see that problem was solved 2 hours ago.