Cannot update tailored audience with TON upload file

[alex_coppens]

Hello,

I am not sure which part of the forum this belongs to, I already posted it in the REST API section. I am having trouble updating a tailored audience with the filename I get back from the TON API. Here is what the error looks like:

[INFO ] irb:13617 - File stored at: /1.1/ton/data/ta_partner/152397343/GejT4XUdexh2sOF.txt?resumable=true&resumeId=710599
[INFO ] irb:13617 - --- Assigning file to twitter tailored audience
Doing request:post accounts/2iqejj/ tailored_audience_changes {:input_file_path=>"/1.1/ton/data/ta_partner/152397343/GejT4XUdexh2sOF.txt?resumable=true&resumeId=710599", :tailored_audience_id=>"u9g0", :operation=>"ADD"} URL:https://ads-api.twitter.com/0/accounts/2iqejj/tailored_audience_changes
TwitterAds::AdsError: [{"code"=>"INVALID_PARAMETER", "message"=>"Expected valid file path, got \"/ta_partner/152397343/GejT4XUdexh2sOF.txt?resumable=true&resumeId=710599\" for input_file_path", "parameter"=>"input_file_path"}]

UPDATE: I tested with a single file upload, I get no error. The above happens when I do it in chunks.

[andypiper]

The TON API is most relevant to the Ads API at present, so this is the right place. Hopefully one of the team can take a look soon!

[alex_coppens]

Hello, this error disappeared for a while but it reappeared very recently. Do you have any information on this issue? The problem seem to only occur when using the multi-chunk upload. It’s quite important as we rely on it on our production platform:

2015-10-27 03:39:11.971158500 [INFO ] twitter-upload:30787 - Uploading chunk #56 (bytes 58720256 to 58790938 out of 58790939
2015-10-27 03:39:12.650864500 [INFO ] twitter-upload:30787 - File stored at: /1.1/ton/data/ta_partner/152397343/iYs0YiUNut9JcxI.txt?resumable=true&resumeId=333011
2015-10-27 03:39:12.650888500 [INFO ] twitter-upload:30787 - — Assigning file to twitter tailored audience
2015-10-27 03:39:13.089153500 [ERROR] twitter-upload:30787 - TwitterAds::AdsError: INVALID_PARAMETER - Error uploading tailored audience

Let me know if you need more details.

[brandonmblack]

Does the error occur when using the ton-upload script for a multi-chunk upload?

If so, can you reproduce the issue with the ton-upload script and show the full request / response output via the --trace flag (you can omit the binary data)?

[alex_coppens]

Hello brandon,

Thanks for your reply, I think I found part of the issue even though I havent fully tested the bug fix I made:

I had audiences that were deleted on the twitter ads site but not locally, so when I tried to re-upload its content with the twitter audience id, it would return the error I described above. So now I always check that a placeholder exists before uploading. I will try to see if I can reproduce the error with trace enabled.

Alex

[alex_coppens]

Yes it was with the ton-upload script for a multi-chunk upload.

[alex_coppens]

I am getting the following on last attempt:

-> "HTTP/1.1 429 Too Many Requests\r\n"
-> "connection: close\r\n"
-> "content-length: 20\r\n"
-> "content-type: text/plain;charset=utf-8\r\n"
-> "date: Tue, 10 Nov 2015 21:14:19 GMT\r\n"
-> "server: tsa_b\r\n"
-> "set-cookie: guest_id=v1%3A144719005953060853; Domain=.twitter.com; Path=/; Expires=Thu, 09-Nov-2017 21:14:19 UTC\r\n"
-> "strict-transport-security: max-age=631138519\r\n"
-> "x-connection-hash: cf52b54048ab6347481a2d9b01a1d0b3\r\n"
-> "x-rate-limit-limit: 50\r\n"
-> "x-rate-limit-remaining: 0\r\n"
-> "x-rate-limit-reset: 1447190549\r\n"
-> "x-response-time: 11\r\n"
-> "x-tsa-request-body-time: 321\r\n"
-> "\r\n"
reading 20 bytes...
-> ""
-> "Rate limit exceeded\n"
read 20 bytes
Conn close
[INFO ] twitter-upload:2871 - File stored at: /1.1/ton/data/ta_partner/152397343/h42KkOKF96Uan1J.txt?resumable=true&resumeId=527777
[INFO ] twitter-upload:2871 - --- Assigning file to twitter tailored audience
Doing request:post accounts/2iqejj/ tailored_audience_changes {:input_file_path=>"/1.1/ton/data/ta_partner/152397343/h42KkOKF96Uan1J.txt?resumable=true&resumeId=527777", :tailored_audience_id=>"148w1", :operation=>"ADD"} URL:https://ads-api.twitter.com/0/accounts/2iqejj/tailored_audience_changes
[ERROR] twitter-upload:2871 - TwitterAds::AdsError: INVALID_PARAMETER - Error uploading tailored audience 6937b5b0-6a18-0133-97a3-080027880ca6
Doing request:get accounts/2iqejj/tailored_audiences/148w1   URL:https://ads-api.twitter.com/0/accounts/2iqejj/tailored_audiences/148w1

[alex_coppens]

I am also getting this on intermediate requests:

-> "HTTP/1.1 400 Bad Request\r\n"
-> "cache-control: no-cache\r\n"
-> "connection: close\r\n"
-> "content-length: 0\r\n"
-> "date: Tue, 10 Nov 2015 21:20:11 GMT\r\n"
-> "server: tsa_b\r\n"
-> "set-cookie: guest_id=v1%3A144719041188757197; Domain=.twitter.com; Path=/; Expires=Thu, 09-Nov-2017 21:20:11 UTC\r\n"
-> "strict-transport-security: max-age=631138519\r\n"
-> "x-connection-hash: ca66780c0e4a613787342ace7bba10d9\r\n"
-> "x-content-type-options: nosniff\r\n"
-> "x-response-time: 27\r\n"
-> "x-tsa-request-body-time: 1668\r\n"
-> "\r\n"
reading 0 bytes...

The requests look like this:

[INFO ] twitter-upload:2871 - Uploading chunk #70 (bytes 73400320 to 74448895 out of 218255244
opening connection to ton.twitter.com...
opened
<- "PUT https://ton.twitter.com HTTP/1.1\r\nAccept: */*\r\nUser-Agent: OAuth gem v0.4.7\r\nContent-Type: text/plain\r\nContent-Length: 1048576\r\nContent-Range: bytes 73400320-74448895/218255244\r\nAuthorization: OAuth oauth_body_hash=\"02..G4%3D\
", oauth_consumer_key=\"YOi6D....Opb\", oauth_nonce=\"R..t4\", oauth_signature=\"D\", oauth_signature_method=\"HMAC-SHA1\", oauth_timestamp=\"1447190411\", oauth_token=\"15...z0\", oauth_version=\"1.0\"\r\nConnection: close\r\nHost: ton.twitter.com\r\n\r\n"
<- "d6839621

Update: Now I am getting permanent redirects:

-> "HTTP/1.1 308 Permanent Redirect\r\n"
-> "connection: close\r\n"
-> "content-length: 0\r\n"
-> "date: Tue, 10 Nov 2015 21:40:04 GMT\r\n"
-> "range: bytes=0-3145727\r\n"
-> "server: tsa_b\r\n"
-> "set-cookie: guest_id=v1%3A144719160355709640; Domain=.twitter.com; Path=/; Expires=Thu, 09-Nov-2017 21:40:04 UTC\r\n"
-> "strict-transport-security: max-age=631138519\r\n"
-> "x-connection-hash: 8756b608bd3c015c29137eaf755e3c2e\r\n"
-> "x-content-type-options: nosniff\r\n"
-> "x-rate-limit-limit: 50\r\n"
-> "x-rate-limit-remaining: 39\r\n"
-> "x-rate-limit-reset: 1447192141\r\n"
-> "x-response-time: 609\r\n"
-> "x-tsa-request-body-time: 1609\r\n"
-> "\r\n"
reading 0 bytes...
-> ""
read 0 bytes

[alex_coppens]

We did some tests today and the problem is really that we try to upload too many chunks at once.

-> “x-rate-limit-limit: 50\r\n”
-> “x-rate-limit-remaining: 32\r\n”
-> “x-rate-limit-reset: 1447774444\r\n”

with the remaining part eventually going to zero. We will implement a limiter on our side.

[brandonmblack]

Thanks for the updates @alex_coppens. Great to hear you were able to make progress.

Your requests actually look correct, but it looks like you’re just hitting rate limits in both the TON API and the Ads API due to your current level of access. You are unfortunately pretty limited on the size and speed of requests you can make for tailored audience uploads at the developer level.

You can read more about Ads API rate limits here:
https://dev.twitter.com/ads/basics/rate-limiting

Please note that the TON API uses blanket 400s in all client errors including rate limited requests.

[alex_coppens]

@brandonmblack Thanks for the link, somehow we missed that part. For now we are using single chunk uploads (we raised the limit to 256Mb) as much as possible since we have a pretty solid connection to Twitter. We also implemented some rate limit checks to wait until next reset if our request budget goes to zero. That should solve a lot of problems we had.

[brandonmblack]

@alex_coppens sounds good. Please follow up here again if you have more trouble.

[brandonmblack]