Question about duplicates in location data in V1 & V2 twitter API

chris_august7 · 2017-10-25 18:12:50 UTC

Hi,

We have been running queries to ingest new location targeting data. There seemingly are duplicate records. Can you explain why there are duplicates? An example is below:

{
    "data": [
        {
            "name": "Los Angeles, Los Angeles CA, CA, USA",
            "country_code": "US",
            "location_type": "CITIES",
            "targeting_value": "3b77caf94bfc81fe",
            "targeting_type": "LOCATION"
        },
        {
            "name": "East Los Angeles, Los Angeles CA, CA, USA",
            "country_code": "US",
            "location_type": "CITIES",
            "targeting_value": "67571a7baaa5906b",
            "targeting_type": "LOCATION"
        },
        {
            "name": "Los Angeles, US",
            "country_code": "US",
            "location_type": "CITIES",
            "targeting_value": "019940ae78c7b3bc",
            "targeting_type": "LOCATION"
        }
    ],
    "request": {
        "params": {
            "location_type": "CITIES",
            "q": "los angeles"
        }
    },
    "next_cursor": null
}

chris_august7 · 2017-10-25 16:31:00 UTC

Also… the results above are inconsistent with results we get from queries to synch down all locations. Note that “targeting_value” is the same in both cases below, whereas for the two “Los Angeles” cities in the query above, they are different.

GET request on url https://ads-api.twitter.com/2/targeting_criteria/locations?cursor=XYZ returns

{
            "name": "Los Angeles, US",
            "country_code": "US",
            "location_type": "CITIES",
            "targeting_value": "3b77caf94bfc81fe",
            "targeting_type": "LOCATION"
        },

while GET https://ads-api.twitter.com/2/targeting_criteria/locations?location_type=CITIES&q=los angeles returns

  {
            "name": "Los Angeles, Los Angeles CA, CA, USA",
            "country_code": "US",
            "location_type": "CITIES",
            "targeting_value": "3b77caf94bfc81fe",
            "targeting_type": "LOCATION"
        }

chris_august7 · 2017-10-25 18:13:46 UTC

Here is some more information based on what we’ve synched down:

SELECT * FROM tw_location WHERE name ILIKE '%los angeles%';

 targeting_value  | location_type |             name              | iso_abbreviation | un_abbreviation
------------------+---------------+-------------------------------+------------------+-----------------
 e4315a8602024dcb | CITY          | Los Angeles CA, US            |                  |
 178ef2da29c52481 | CITY          | Torrecilla de los Angeles, ES |                  |
 67571a7baaa5906b |               | East Los Angeles, US          |                  |
 ea9bfbd43c93400f |               | Lake Los Angeles, US          |                  |
 019940ae78c7b3bc |               | Los Angeles, US               |                  |
 3b77caf94bfc81fe |               | Los Angeles, US               |                  |
(6 rows)

3b77caf94bfc81fe = CA
019940ae78c7b3bc = TX

based on comments via email from @juanshishido

this is in the V1 API

juanshishido · 2017-10-25 22:50:39 UTC

Thanks for raising this, @chris_august7. For visibility, something similar has been posted here.

We appreciate the details you’ve provided. For the following Los Angeles targeting options, the first refers to the city in California and the second to Texas, as you mentioned.

{  
  "name":"Los Angeles, Los Angeles CA, CA, USA",
  "country_code":"US",
  "location_type":"CITIES",
  "targeting_value":"3b77caf94bfc81fe",
  "targeting_type":"LOCATION"
},
{  
  "name":"Los Angeles, US",
  "country_code":"US",
  "location_type":"CITIES",
  "targeting_value":"019940ae78c7b3bc",
  "targeting_type":"LOCATION"
}

We’re looking to update these name values so that they correctly return the state information.

For cases where the name is different for the same targeting_value—e.g., with 3b77caf94bfc81fe—know that the targeting_value will always correspond to a single entity. In other words, 3b77caf94bfc81fe will always refer to the city of Los Angeles in California, even if sometimes it’s returned as “Los Angeles, US.”

Again, looking to include the state information to make this clearer.

We don’t have a timeline yet for this fix, but will update this thread (and the other one) once we have more information.

Thanks!

thizanetti87 · 2017-10-26 20:02:33 UTC

@juanshishido When calling GET request on url https://ads-api.twitter.com/2/targeting_criteria/locations?cursor=XYZ at some point in the past in v1 it returned 15242 records with location_type NULL

twcachedb=> SELECT count(*) FROM tw_location where location_type is null;
 count
-------
 15242
(1 row)

It seems that those locations in v2 have now been changed from location_type null to location_type CITIES. Here are some examples of the data difference.

v1

twcachedb=> SELECT * FROM tw_location WHERE name ILIKE '%los angeles%';
 targeting_value  | location_type |             name              | iso_abbreviation | un_abbreviation
------------------+---------------+-------------------------------+------------------+-----------------
 e4315a8602024dcb | CITY          | Los Angeles CA, US            |                  |
 178ef2da29c52481 | CITY          | Torrecilla de los Angeles, ES |                  |
 67571a7baaa5906b |               | East Los Angeles, US          |                  |
 ea9bfbd43c93400f |               | Lake Los Angeles, US          |                  |
 019940ae78c7b3bc |               | Los Angeles, US               |                  |
 3b77caf94bfc81fe |               | Los Angeles, US               |                  |
(6 rows)

twcachedb=> SELECT * FROM tw_location WHERE name ILIKE '%alaska%';
 targeting_value  | location_type |     name     | iso_abbreviation | un_abbreviation
------------------+---------------+--------------+------------------+-----------------
 07179f4fe0500a32 | REGION        | Alaska, US   |                  |
 001b46b528e432f4 |               | Alaska, US   |                  |
 00801d005116e0a4 |               | Onalaska, US |                  |
 016c69727153270b |               | Onalaska, US |                  |
 2cd630bad4aa6e0b |               | Unalaska, US |                  |
(5 rows)

twcachedb=> SELECT * FROM tw_location WHERE name ILIKE '%brazil%';
 targeting_value  | location_type |    name    | iso_abbreviation | un_abbreviation
------------------+---------------+------------+------------------+-----------------
 c04138d310229f7e |               | Brazil, US |                  |
 1b107df3ccc0aaa1 | COUNTRY       | Brazil     | BR               | BRA
(2 rows)
```

## v2 - same records in v2
```
twcachedb=> SELECT * FROM tw_location WHERE name ILIKE '%los angeles%';
 targeting_value  | location_type |             name              | iso_abbreviation | un_abbreviation
------------------+---------------+-------------------------------+------------------+-----------------
 67571a7baaa5906b | CITIES        | East Los Angeles, US          |                  |
 ea9bfbd43c93400f | CITIES        | Lake Los Angeles, US          |                  |
 e4315a8602024dcb | METROS        | Los Angeles CA, US            |                  |
 019940ae78c7b3bc | CITIES        | Los Angeles, US               |                  |
 3b77caf94bfc81fe | CITIES        | Los Angeles, US               |                  |
 178ef2da29c52481 | METROS        | Torrecilla de los Angeles, ES |                  |
(6 rows)

twcachedb=> SELECT * FROM tw_location WHERE name ILIKE '%alaska%';
 targeting_value  | location_type |     name     | iso_abbreviation | un_abbreviation
------------------+---------------+--------------+------------------+-----------------
 07179f4fe0500a32 | REGIONS       | Alaska, US   |                  |
 001b46b528e432f4 | CITIES        | Alaska, US   |                  |
 00801d005116e0a4 | CITIES        | Onalaska, US |                  |
 016c69727153270b | CITIES        | Onalaska, US |                  |
 2cd630bad4aa6e0b | CITIES        | Unalaska, US |                  |
(5 rows)

twcachedb=> SELECT * FROM tw_location WHERE name ILIKE '%brazil%';
 targeting_value  | location_type |    name    | iso_abbreviation | un_abbreviation
------------------+---------------+------------+------------------+-----------------
 c04138d310229f7e | CITIES        | Brazil, US |                  |
 1b107df3ccc0aaa1 | COUNTRIES     | Brazil     | BR               | BRA
(2 rows)
```

juanshishido · 2018-01-04 05:33:00 UTC

@chris_august7 and @thizanetti87: Thanks again for bringing this to our attention. This issue is now fixed.