We used to be able to expect Async Stats jobs to finish in around 20s with the outliers completing at about a minute or two. Starting somewhere around 8/19 we noticed that there are times where jobs will take more than 15 minutes, and sometimes not ever complete. Occasionally it goes back to being reasonable with like 2 minute completions (this used to be about the max).
Is there any known issues about this, or is the api just degrading as more data exists?
We do a lot of storing of the Twitter data, but the numbers even for months ago can change randomly, so we attempt to keep our data in sync with Twitter. The fact that the jobs are taking so long and we are limited to 100 running is hindering our ability to complete this since we manage so many accounts.
It would be awesome if there was some issue that is being worked on, or some way we can change our calls to improve the api. Or if the 99% accurate by 3 days and completely locked by 14 was actually true. Then we would not have to refresh ourselves.