Ads API async job returns all empty "id_data" fields

[ribitskiyb]

I use twitter4j-ads (Java) to pull ad campaigns performance statictics.

At first a valid data seemed to be returned, but recently I started to receive empty data_id field for every entity in resulting json. I’ve also assured that the data is present for the periods I’m trying to get it via API using Ads Manager web UI.

Request parameters are the following:

• account_id = 8vjec
• entity = CAMPAIGN
• segmentation_type = LOCATIONS
• metric_groups = BILLING, ENGAGEMENT, WEB_CONVERSION
• granularity = DAY
• placement = ALL_ON_TWITTER

I repeat this request several times to get data for all entities (campaigns, in my case) in the account, entity_ids are in base-36. Time period of each request is a single date.

Please help to understand what I may be doing wrong.

[JBabichJapan]

Hey @ribitskiyb does this happen for only certain cases or all requests? If you can paste a snippet of code it might help to narrow down otherwise I would recommend to see if you get the same empty result via twurl (it might be a data issue then unrelated to SDK).

[ribitskiyb]

@JBabichJapan,

full application’s code is in github repo, initial request to the API is made in createAsyncJob() method, which in turn uses twitter4j-ads's class to actually execute request (a lot of code there , didn’t exemine it that thoroughly yet).

Thanks for check via twurl suggestion, I’ll try it. Also any further input from you is highly welcome.

[ribitskiyb]

Hey @JBabichJapan,

I’ve tried manually downloading the report using twurl – got the same result, empty id_data for all entities.

Initial request initiating async job:
twurl -X POST -H ads-api.twitter.com "/2/stats/jobs/accounts/8vjec?start_time=2016-01-01&end_time=2016-01-02&entity=CAMPAIGN&entity_ids=82sjq,8p3cm,78w3i,7icdj,78w3p,7comq,78w3n,78w3s,78w55,78w3t,78w56,78w3q,78w3r,78w59,78w3x,78w3u,78w57,8um78,8tgwa,78w3z&granularity=DAY&metric_groups=BILLING,ENGAGEMENT,WEB_CONVERSION&placement=ALL_ON_TWITTER&segmentation_type=LOCATIONS"

[juanshishido]

@ribitskiyb: As a good practice during debugging, let’s focus on a single entity—82sjq—and a single metric group—ENGAGEMENT.

$ twurl -H ads-api.twitter.com "/2/stats/accounts/8vjec?start_time=2016-01-01&end_time=2016-01-02&entity=CAMPAIGN&entity_ids=82sjq&granularity=DAY&metric_groups=ENGAGEMENT&placement=ALL_ON_TWITTER"
{
  "data_type": "stats",
  "time_series_length": 1,
  "data": [
    {
      "id": "82sjq",
      "id_data": [
        {
          "segment": null,
          "metrics": {
            "impressions": null,
            "tweets_send": null,
            "qualified_impressions": null,
            "follows": null,
            "app_clicks": null,
            "retweets": null,
            "likes": null,
            "engagements": null,
            "clicks": null,
            "card_engagements": null,
            "poll_card_vote": null,
            "replies": null,
            "url_clicks": null,
            "carousel_swipes": null
          }
        }
      ]
    }
  ],
  "request": {
    "params": {
      "start_time": "2016-01-01T05:00:00Z",
      "segmentation_type": null,
      "entity_ids": [
        "82sjq"
      ],
      "end_time": "2016-01-02T05:00:00Z",
      "country": null,
      "placement": "ALL_ON_TWITTER",
      "granularity": "DAY",
      "entity": "CAMPAIGN",
      "platform": null,
      "metric_groups": [
        "ENGAGEMENT"
      ]
    }
  }
}

No data, as you mentioned.

However, when we take a look at that specific campaign, we notice that its start time is 2017-04-11T12:15:00Z.

$ twurl -H ads-api.twitter.com "/2/accounts/8vjec/campaigns/82sjq"
{
  "request": {
    "params": {
      "campaign_id": "82sjq",
      "account_id": "8vjec"
    }
  },
  "data": {
    "name": {name},
    "start_time": "2017-04-11T12:15:00Z",
    "reasons_not_servable": [
      "EXPIRED",
      "FUNDING_PROBLEM"
    ],
    "servable": false,
    "daily_budget_amount_local_micro": 20000000,
    "end_time": null,
    "funding_instrument_id": "nfw5s",
    "duration_in_days": null,
    "standard_delivery": true,
    "total_budget_amount_local_micro": null,
    "id": "82sjq",
    "entity_status": "PAUSED",
    "account_id": "8vjec",
    "frequency_cap": null,
    "currency": "USD",
    "created_at": "2017-04-11T12:15:06Z",
    "updated_at": "2017-05-26T17:10:08Z",
    "deleted": false
  }
}

You’re requesting data for start_time=2016-01-01&end_time=2016-01-02. It is expected that the stats request return all nulls.

[ribitskiyb]

@juanshishido, thanks!

I would only mention that by “empty” id_data I mean not null values but just an empty array, like so
{ "data_type": "stats", "time_series_length": 1, "data": [ { "id": "82sjq", "id_data": [] } ...

Can it happen because of unsupported combination of Metric Group and Segmentation Unit? (e.g. WEB_CONVERSION combined with LOCATIONS)

[juanshishido]

@ribitskiyb: I see. Thanks for the clarification. Let me continue looking. Thanks for your patience.

[juanshishido]

@ribitskiyb: This likely still has to do with the fact that there isn’t any data for this campaign for this time period. Please see the example requests, below.

The synchronous request (note: no segmentation_type yet, but we’ll get there):

$ twurl -H ads-api.twitter.com "/2/stats/accounts/8vjec?start_time=2017-04-11&end_time=2017-04-18&entity=CAMPAIGN&entity_ids=82sjq&granularity=TOTAL&metric_groups=ENGAGEMENT&placement=ALL_ON_TWITTER"
{
  "data_type": "stats",
  "time_series_length": 1,
  "data": [
    {
      "id": "82sjq",
      "id_data": [
        {
          "segment": null,
          "metrics": {
            "impressions": [
              59699
            ],
            "tweets_send": null,
            "qualified_impressions": null,
            "follows": [
              1
            ],
            "app_clicks": null,
            "retweets": null,
            "likes": [
              6
            ],
            "engagements": [
              78
            ],
            "clicks": [
              76
            ],
            "card_engagements": null,
            "poll_card_vote": null,
            "replies": [
              1
            ],
            "url_clicks": [
              14
            ],
            "carousel_swipes": null
          }
        }
      ]
    }
  ],
  "request": {
    "params": {
      "start_time": "2017-04-11T04:00:00Z",
      "segmentation_type": null,
      "entity_ids": [
        "82sjq"
      ],
      "end_time": "2017-04-18T04:00:00Z",
      "country": null,
      "placement": "ALL_ON_TWITTER",
      "granularity": "TOTAL",
      "entity": "CAMPAIGN",
      "platform": null,
      "metric_groups": [
        "ENGAGEMENT"
      ]
    }
  }
}

Now, we make the same request (no segmentation_type yet), using the asynchronous analytics endpoint. (We show the result after downloading the .gzip).

$ twurl -X POST -H ads-api.twitter.com "/2/stats/jobs/accounts/8vjec?start_time=2017-04-11&end_time=2017-04-18&entity=CAMPAIGN&entity_ids=82sjq&granularity=TOTAL&metric_groups=ENGAGEMENT&placement=ALL_ON_TWITTER"
{
  "data_type": "stats",
  "time_series_length": 1,
  "data": [
    {
      "id": "82sjq",
      "id_data": [
        {
          "segment": null,
          "metrics": {
            "impressions": [
              59699
            ],
            "tweets_send": null,
            "qualified_impressions": null,
            "follows": [
              1
            ],
            "app_clicks": null,
            "retweets": null,
            "likes": [
              6
            ],
            "engagements": [
              78
            ],
            "clicks": [
              76
            ],
            "card_engagements": null,
            "poll_card_vote": null,
            "replies": [
              1
            ],
            "url_clicks": [
              14
            ],
            "carousel_swipes": null
          }
        }
      ]
    }
  ],
  "request": {
    "params": {
      "start_time": "2017-04-11T04:00:00Z",
      "segmentation_type": null,
      "entity_ids": [
        "82sjq"
      ],
      "end_time": "2017-04-18T04:00:00Z",
      "country": null,
      "placement": "ALL_ON_TWITTER",
      "granularity": "TOTAL",
      "entity": "CAMPAIGN",
      "platform": null,
      "metric_groups": [
        "ENGAGEMENT"
      ]
    }
  }
}

Now that we’re confident that we can see results for this campaign for this time period, let’s add segmentation_type.

$ twurl -X POST -H ads-api.twitter.com "/2/stats/jobs/accounts/8vjec?start_time=2017-04-11&end_time=2017-04-18&entity=CAMPAIGN&entity_ids=82sjq&granularity=TOTAL&metric_groups=ENGAGEMENT&placement=ALL_ON_TWITTER&segmentation_type=LOCATIONS"
{
  "data_type": "stats",
  "time_series_length": 1,
  "data": [
    {
      "id": "82sjq",
      "id_data": [
        {
          "segment": {
            "segment_name": "Israel",
            "segment_value": "2edb6e240797c549"
          },
          "metrics": {
            "impressions": [
              712
            ],
            "tweets_send": null,
            "qualified_impressions": null,
            "follows": null,
            "app_clicks": null,
            "retweets": null,
            "likes": null,
            "engagements": null,
            "clicks": null,
            "card_engagements": null,
            "poll_card_vote": null,
            "replies": null,
            "url_clicks": null,
            "carousel_swipes": null
          }
        },
        {
          "segment": {
            "segment_name": "Spain",
            "segment_value": "ecdce75d48b13b64"
          },
          "metrics": {
            "impressions": [
              11446
            ],
            "tweets_send": null,
            "qualified_impressions": null,
            "follows": null,
            "app_clicks": null,
            "retweets": null,
            "likes": null,
            "engagements": [
              13
            ],
            "clicks": [
              13
            ],
            "card_engagements": null,
            "poll_card_vote": null,
            "replies": null,
            "url_clicks": [
              3
            ],
            "carousel_swipes": null
          }
        },
        {
          "segment": {
            "segment_name": "Portugal",
            "segment_value": "8198e85105936d3c"
          },
          "metrics": {
            "impressions": [
              3675
            ],
            "tweets_send": null,
            "qualified_impressions": null,
            "follows": null,
            "app_clicks": null,
            "retweets": null,
            "likes": null,
            "engagements": [
              7
            ],
            "clicks": [
              6
            ],
            "card_engagements": null,
            "poll_card_vote": null,
            "replies": [
              1
            ],
            "url_clicks": [
              1
            ],
            "carousel_swipes": null
          }
        },
        {
          "segment": {
            "segment_name": "Greece",
            "segment_value": "2ee7eeaa84dbe65a"
          },
          "metrics": {
            "impressions": [
              5340
            ],
            "tweets_send": null,
            "qualified_impressions": null,
            "follows": null,
            "app_clicks": null,
            "retweets": null,
            "likes": [
              1
            ],
            "engagements": [
              5
            ],
            "clicks": [
              5
            ],
            "card_engagements": null,
            "poll_card_vote": null,
            "replies": null,
            "url_clicks": null,
            "carousel_swipes": null
          }
        },
        {
          "segment": {
            "segment_name": "France",
            "segment_value": "f3bfc7dcc928977f"
          },
          "metrics": {
            "impressions": [
              11092
            ],
            "tweets_send": null,
            "qualified_impressions": null,
            "follows": null,
            "app_clicks": null,
            "retweets": null,
            "likes": [
              2
            ],
            "engagements": [
              12
            ],
            "clicks": [
              12
            ],
            "card_engagements": null,
            "poll_card_vote": null,
            "replies": null,
            "url_clicks": [
              2
            ],
            "carousel_swipes": null
          }
        },
        {
          "segment": {
            "segment_name": "Italy",
            "segment_value": "c799e2d3a79f810e"
          },
          "metrics": {
            "impressions": [
              5948
            ],
            "tweets_send": null,
            "qualified_impressions": null,
            "follows": null,
            "app_clicks": null,
            "retweets": null,
            "likes": [
              1
            ],
            "engagements": [
              13
            ],
            "clicks": [
              13
            ],
            "card_engagements": null,
            "poll_card_vote": null,
            "replies": null,
            "url_clicks": [
              2
            ],
            "carousel_swipes": null
          }
        },
        {
          "segment": {
            "segment_name": "Turkey",
            "segment_value": "682c5a667856ef42"
          },
          "metrics": {
            "impressions": [
              21481
            ],
            "tweets_send": null,
            "qualified_impressions": null,
            "follows": [
              1
            ],
            "app_clicks": null,
            "retweets": null,
            "likes": [
              2
            ],
            "engagements": [
              28
            ],
            "clicks": [
              27
            ],
            "card_engagements": null,
            "poll_card_vote": null,
            "replies": null,
            "url_clicks": [
              6
            ],
            "carousel_swipes": null
          }
        }
      ]
    }
  ],
  "request": {
    "params": {
      "start_time": "2017-04-11T04:00:00Z",
      "segmentation_type": "LOCATIONS",
      "entity_ids": [
        "82sjq"
      ],
      "end_time": "2017-04-18T04:00:00Z",
      "country": null,
      "placement": "ALL_ON_TWITTER",
      "granularity": "TOTAL",
      "entity": "CAMPAIGN",
      "platform": null,
      "metric_groups": [
        "ENGAGEMENT"
      ]
    }
  }
}

I hope this helps clarify. Thanks!