Async job completion time getting worse

[guptamedia]

We used to be able to expect Async Stats jobs to finish in around 20s with the outliers completing at about a minute or two. Starting somewhere around 8/19 we noticed that there are times where jobs will take more than 15 minutes, and sometimes not ever complete. Occasionally it goes back to being reasonable with like 2 minute completions (this used to be about the max).

Is there any known issues about this, or is the api just degrading as more data exists?

We do a lot of storing of the Twitter data, but the numbers even for months ago can change randomly, so we attempt to keep our data in sync with Twitter. The fact that the jobs are taking so long and we are limited to 100 running is hindering our ability to complete this since we manage so many accounts.

It would be awesome if there was some issue that is being worked on, or some way we can change our calls to improve the api. Or if the 99% accurate by 3 days and completely locked by 14 was actually true. Then we would not have to refresh ourselves.

[JBabichJapan]

Hi,

Generally outside of incidents and service degradation periods we definitely do not expect the async jobs to be taking more than 10 mins. Additionally for the ones that never complete there is not really any known issue tracking something like that so I would be curious to see some specific examples from you if you are able to share them.

There are cases where data changes even after 3 days:

• Conversion data based upon the window of either website tag or app event tag (i.e. if someone converts and purchases something, depending on window it can be 5 days later and suddenly increment one of the stats values)
• Organic data of course might change

For billing data since it’s so sensitive per hour of day you might also see data change depending on how you are syncing, like including or excluding hours / double counting them, so it’s something to be careful about. If you see problems that are easy to repro I would definitely recommend you to submit a new post about it with as much details as possible.

Thanks,

John

[JBabichJapan]

Just to post an update here as well - as I mentioned in Analytics Jobs Taking a Long Time to Complete we have identified some moderate degradation but do not expect it to last. I would expect the performance to improve soon but if it doesn’t please look out for more updates.