Large delays in getting metric data from previous day (Analytics API async reports)


#1

Hi,

We run scheduled Analytics API async report jobs at 6:00 AM EDT/EST to load metrics from the previous day.
Since early January, 2019 we have noticed that the previous day’s metric numbers retrieved (impressions, clicks, etc.) are either 0 or incomplete when our scheduled reports run at that time.
Note that we’ve been running these reports daily for years and previously the discrepancies due to late/incomplete data have been few and far between.
Has something changed which causes these delays?
Can this be fixed?
Also, how can we better ensure that we are getting complete numbers for the previous day?
We’ve temporarily moved our schedule up to run at 11:00 AM to try to mitigate this problem but even at that time the data is not all available.

We retrieve data for multiple accounts and do not think this is an issue specific to one particular account, but to help you investigate we are definitely seeing it on account qfdk20

Thanks


#2

To give more context as to the severity of the delays we are seeing: data for 2019-01-15 was still not completely being returned at 9:00 AM on 2019-01-17 (33 hours later)


#3

Can anyone from Twitter answer this?
We feel that waiting more than 6-8 hours for data is a little excessive and far exceeds what we’ve come to expect from the Twitter Ads Analytics APIs over the past 3 or 4 years that we’ve been using them.


#4

Thanks for the follow up here, @emanuelp. We just started looking at the account you referenced (thanks for providing that).

I’m seeing the data update in real-time for today. Tried both sync and async. Here are just two examples, about 20 seconds apart.

$ date -u; twurl -H ads-api.twitter.com "/4/stats/accounts/qfdk20?entity=ACCOUNT&entity_ids=qfdk20&start_time=2019-01-22&end_time=2019-01-23&metric_groups=ENGAGEMENT&granularity=TOTAL&placement=ALL_ON_TWITTER"
Tue Jan 22 15:47:05 UTC 2019
{
  "data_type": "stats",
  "time_series_length": 1,
  "data": [
    {
      "id": "qfdk20",
      "id_data": [
        {
          "segment": null,
          "metrics": {
            "impressions": [
              51677
            ],
            "follows": [
              4
            ],
            "retweets": null,
            "unfollows": null,
            "likes": [
              9
            ],
            "engagements": [
              358
            ],
            "replies": null
          }
        }
      ]
    }
  ],
  "request": {
    "params": {
      "start_time": "2019-01-22T05:00:00Z",
      "segmentation_type": null,
      "entity_ids": null,
      "end_time": "2019-01-23T05:00:00Z",
      "country": null,
      "placement": "ALL_ON_TWITTER",
      "granularity": "TOTAL",
      "entity": "ACCOUNT",
      "platform": null,
      "metric_groups": [
        "ENGAGEMENT"
      ]
    }
  }
}
$ date -u; twurl -H ads-api.twitter.com "/4/stats/accounts/qfdk20?entity=ACCOUNT&entity_ids=qfdk20&start_time=2019-01-22&end_time=2019-01-23&metric_groups=ENGAGEMENT&granularity=TOTAL&placement=ALL_ON_TWITTER"
Tue Jan 22 15:47:26 UTC 2019
{
  "data_type": "stats",
  "time_series_length": 1,
  "data": [
    {
      "id": "qfdk20",
      "id_data": [
        {
          "segment": null,
          "metrics": {
            "impressions": [
              51768
            ],
            "follows": [
              4
            ],
            "retweets": null,
            "unfollows": null,
            "likes": [
              9
            ],
            "engagements": [
              358
            ],
            "replies": null
          }
        }
      ]
    }
  ],
  "request": {
    "params": {
      "start_time": "2019-01-22T05:00:00Z",
      "segmentation_type": null,
      "entity_ids": null,
      "end_time": "2019-01-23T05:00:00Z",
      "country": null,
      "placement": "ALL_ON_TWITTER",
      "granularity": "TOTAL",
      "entity": "ACCOUNT",
      "platform": null,
      "metric_groups": [
        "ENGAGEMENT"
      ]
    }
  }
}

In ads manager, I’m also not seeing a delay in data. For example, to see this issue, I would expect much lower numbers for the previous day, but I’m not seeing that. All the bars prior to today are relatively consistent.

We will keep monitoring this, though.


#5

We’re still making calls to the v3 API like so: https://ads-api.twitter.com/3/stats

Also, we are retrieving stats for each promoted tweet and not at the account level. i.e.

    {
      "account_id": "qfdk20",
      "start_time": "2019-01-22T05:00:00Z",
      "end_time": "2019-01-23T05:00:00Z",
      "segmentation_type": "PLATFORMS",
      "placement": "ALL_ON_TWITTER",
      "granularity": "DAY",
      "entity": "PROMOTED_TWEET",
      "id":<REDACTED>,
      "metric_groups": [
        "BILLING",
        "VIDEO",
        "MOBILE_CONVERSION",
        "ENGAGEMENT",
        "MEDIA",
        "WEB_CONVERSION"
      ]
    }

Might this make a difference in the availability of the data?


#6

@juanshishido We are still experiencing this problem… any news on this?


#7

@juanshishido We had mitigated this by reloading the previous 3 days every morning.
However, I have just re-checked the data loaded this morning for 2019-02-12 and it does appear to line up with impressions/click totals showing in the Twitter Ads Manager UI.

Was something fixed on your end?
I’d like this confirmed before I consider changing our code back to only loading the previous day.