Uploading data to TON API fails using PHP cURL

[hector_borras]

Thanks @majoritasdev,

Buy how could I do that?

Everywhere I search about how to send the data with PHP cURL says that I should use:
curl_setopt($ch, CURLOPT_POSTFIELDS, $rawData) ;

Thanks

[hector_borras]

Thanks @JBabichJapan , First of all I’m trying to upload a TA with only one record, not using the chunked upload (I’ll cover it when I acomplish to upload a simple file)

I’m following this because I feel more comfortable with python:

twitterdev/twitter-python-ads-sdk/blob/master/twitter_ads/http.py#L194

But in the end, the Request object send data with a body parameter and after it, the lib sends it as
data = self.options.get('body', None)

How can I check if I have the correct permissions for this endpoint/api?

Thanks!

[majoritasdev]

@hector_borras: try this: http://programmers.stackexchange.com/questions/199580/how-can-i-send-data-without-postfields-in-php-curl#199586

[hector_borras]

I’ll try, thanks!

[hector_borras]

I’ve readed that but reading this code:
curl_setopt($ch, CURLOPT_HTTPHEADER, array( 'Content-Type: application/json', 'Content-Length: ' . strlen($data_string)) );

and this one:
`
$data = array(“name” => “Hagrid”, “age” => “36”);
$data_string = json_encode($data);

$ch = curl_init(‘http://api.local/rest/users’);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, “POST”);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data_string);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
‘Content-Type: application/json’,
'Content-Length: ’ . strlen($data_string))
);

$result = curl_exec($ch);
`

It’s more or less what I’m doing, sending it as:

curl_setopt($ch, CURLOPT_POSTFIELDS, $data_string);

But, wait, maybe the thing is replace:

curl_setopt($ch, CURLOPT_POST, 1); --> curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");

Is it a big difference?

Thanks!

[majoritasdev]

I don’t know, maybe it does. Try to see if it makes any difference.

[Blackburn29]

@hector_borras The code on my branch on Git is based on the ruby script that @JBabichJapan posted above. Still having the same 403 errors. I think this has to do with the postfields in cURL being formatted incorrectly

[Blackburn29]

I’ve used the example post above to initalize a multi-chunk upload for a 31MB file of sha256 hashed emails with the following headers:

        [
            'X-TON-Content-Type'   => 'text/plain',
            'X-TON-Content-Length' => 32500000,
            'X-TON-Expires'        => 'Tue, 12 Jul 2016 15:26:11 GMT',
            'Content-Length'       => 0,
            'Content-Type'         => 'text'plain',
        ]

and I still get a 403 forbidden error. I dont see this to be an authentication error either, as my PHP and TWURL requests are sending the exact same Authorization: header (except twurl has the oauth_body_hash)

Twurl (Working, getting 201 Created):

Authorization:
OAuth oauth_body_hash=“ZmqMusMfLNliIT45yUqjwqfwUQ0%3D”,
oauth_consumer_key=“xx”,
oauth_nonce=“qrLoitjRvHgEwdvZi40Xo2CLImOPXXsYzICB4gi3k8”,
oauth_signature=“xx”,
oauth_signature_method=“HMAC-SHA1”,
oauth_timestamp=“1467732784”,
oauth_token=“xx”,
oauth_version=“1.0”

PHP: (Failing, 403 Forbidden)

Authorization:
oauth_consumer_key=“xx”,
oauth_nonce=“13d4b96a5e21615ec1b9d4877051b220”,
oauth_signature=xx"
oauth_signature_method=“HMAC-SHA1”,
oauth_timestamp=“1467732371”,
oauth_token=“xx”,
OAuth oauth_version=“1.0”,

[hector_borras]

I don’t know what could be happening.

I’ve downloaded twitterads-python-sdk and debuged the creation of a TA (it works and I can see the audience in the dashboard).

Those are the headers:

`
POST /1.1/ton/bucket/ta_partner HTTP/1.1
Host: ton.twitter.com
Content-Length: 27
Accept-Encoding: gzip, deflate
x-ton-expires: Fri, 15 Jul 2016 18:24:14 GMT
Accept: /
user-agent: twitter-ads version: 1.1.0 platform: Python 2.7 (CPython/linux2)
Connection: keep-alive
content-type: text/plain
Authorization: OAuth oauth_nonce=“55603487433575853881467743056”, oauth_timestamp=“1467743056”, oauth_version=“1.0”, oauth_signature_method=“HMAC-SHA1”, oauth_consumer_key=“CONSUMER”, oauth_token=“TOKEN”, oauth_signature=“SIGNATURE”

hborrasaleixandre@gmail.com
`

Headers on my request:

POST /1.1/ton/bucket/ta_partner HTTP/1.1 Host: ton.twitter.com User-Agent: TwitterAds SDK v0.1.0 Accept-Encoding: gzip, deflate Accept: */* Authorization: OAuth oauth_version="1.0", oauth_nonce="494db26331f46a46dfd73211d5ef5726", oauth_timestamp="1467745832", oauth_consumer_key="CONSUMER", oauth_token="TOKEN", oauth_signature_method="HMAC-SHA1", oauth_signature="SIGNATURE" x-ton-expires: Fri, 15 Jul 2016 19:10:32 GMT content-type: text/plain Connection: keep-alive Content-Length: 27

Any guess?

[Blackburn29]

The only thing I can think of this something in the way the cURL works through PHP and that the data is not being sent over in the body correctly. Other than that, I’m out of ideas @hector_borras

[JBabichJapan]

Hi folks,

I checked out the php-sdk branch with ton_upload support and tried to debug this, as far as I can tell from diffing with successful run with our Ruby script the major difference is that PHP SDK is not outputting the oauth_body_hash field. Especially because the post contains a payload I wouldn’t be surprised to see that this is required to make sure the payload hasn’t been tampered with.

I would give that a look, I can find some articles mentioning this related to PHP libraries:

Building a body signed oauth xml request for LTI Outcomes service using pecl oauth

Let me know if this solves the issue!

Thanks,

John

[hector_borras]

Thank you ver much @JBabichJapan for your effort.

Searching about an alternative to cURL, I’ve found pecl_http, with this article:

Three Ways to Make a POST Request from PHP

looks similar to your post.

This afternoon (I’m in spain) I’ll check it, hopefully it will work!

Thanks again!

[Blackburn29]

@hector_borras you should look into

http://docs.guzzlephp.org/en/latest/overview.html

This is a great library. I use it all the time

[hector_borras]

I’ll try with this one too!!

I’ll be really happy if I can finish this today!

[hector_borras]

OMG!

EUREKA!!!

PS:

Well, after the euphoria,

The problem was the oauth_body_hash, for cURL is a required parameter. You need to encode base64 the result of encode in sha1 the content.

'oauth_body_hash' => base64_encode(sha1($content));

After that, it works. I’ll see when I’ll need to split the content in different chunks.

Thanks to everybody!!!

[hector_borras]

Anyway it would be good if this documentation will be updated about a mandatory oauth_body_hash in tailored audiences upload.

https://dev.twitter.com/rest/ton/single-chunk

https://dev.twitter.com/rest/ton/resumable

[Blackburn29]

@JBabichJapan @hector_borras @majoritasdev reviving this since I’m having new issues.

I can successfully upload files via the single chunk upload API, however when issuing multi-chunk requests, I am getting 400 error responses AFTER initialization when trying to upload the first chunk.

Heres an example of my request:

PUT /1.1/ton/bucket/ta_partner/340083909/nHbzAQ6zrmIp9FY?resumable=true&resumeId=317889 HTTP/1.1
Expect: 100-Continue
User-Agent: GuzzleHttp/6.2.0 curl/7.35.0 PHP/7.0.8-4+deb.sury.org~trusty+1
Host: ton.twitter.com
Content-Type: text/comma-separated-values
Content-Range: bytes 0-1048575/32500000
oauth_body_hash: YmZiYzRiYTRjYTdlMmFhODEyMDFmMjhjNzkyZDMwZmRkZTQyMTgxZg==
Authorization: OAuth oauth_consumer_key=“X”, oauth_nonce=“cbc3c62f4826a1fa5257bc243f9953b32056a9c0”, oauth_signature=“ksSBxG3PrOXGnxOgOjyLIeG7mzY%3D”, oauth_signature_method=“HMAC-SHA1”, oauth_timestamp=“1468873554”, oauth_token=“X”, oauth_version="1.0"
Content-Length: 1048576

< HTTP/1.1 100 Continue

• We are completely uploaded and fine
  < HTTP/1.1 400 Bad Request

I have compared my headers to the values of the ton_upload ruby script and they are an exact match…
(I also include the oauth_body_hash as mentioned above)

Uploading file in multiple chunks…
Uploading chunk 1 (0-1048575/32500000)…
{“Content-Type”=>“text/comma-separated-values”, “Content-Length”=>1048576, “Content-Range”=>“bytes 0-1048575/32500000”}

Any ideas here?

[hector_borras]

Hey @Blackburn29, don’t worry.

I made a lot of improvements today and now I’m able to upload chunked files too!!!

I need to check if the list that I’ve uploaded one hour ago is correct (waiting for Twitter process) And after that I can push the branch.

Thanks!!!

[Blackburn29]

@hector_borras do you mind explaining what you did differently for chunked files vs single uploads? I’m testing some stuff out on my own code base for this using Guzzle and would like to at least have an understanding of what I’m doing wrong.

[hector_borras]

First of all, I don’t use Guzzle, I make raw cURL requests.

I think your problem is with oauth_body_hash, it should be insidhe the Authorization header like oauth_signature or oauth_version