Direct Message API (message encoding)

[Q_Test_1]

Hi everyone,

I am having a couple of issues when trying to use the Direct Message endpoint (https://api.twitter.com/1.1/direct_messages/new.json) from a Java application. Before anyone recommends it, I do not have the option to use any libraries to make the call, I must do this directly myself.

For both issues, the cause appears to be around the encoding of the actual message itself. I either encode correctly (as far as I can tell) but then authorization fails, or I encode incorrectly and authorization succeeds (but then the message sent is not decoded correctly).

In both cases I want to send the string “This is a:test” (yes, with semi-colon).
In both cases my authentication (“Authorization” header) has not changed. Only the encoding of the message parameter (“text”) has changed.

First Issue - Correct encoding, authorization fails
If I encode the above string to “This%20is%20a%3Atest” then I receive 401 (“Authorization Required”) error.

Second Issue - Incorrect encoding, authorization succeeds
If I encode the above string to “This%2520is%2520a%253Atest” (effectively encoding it twice) then the Direct message is sent, but the recipient receives “This%20is%20a%3Atest”. This appears to have been decoded once (which is what I would expect).

Has anyone seen this before? Does anybody have a pure Java example of hitting this endpoint?

Any help would be very much appreciated.
Thanks

[abraham]

In both cases my authentication (“Authorization” header) has not changed. Only the encoding of the message parameter (“text”) has changed.

Perhaps you mean you are not changing the credentials used but changing text will change the oauth_signature plus each request should have a new oauth_timestamp and oauth_nonce.

I would recommend looking at the source for twitter4j and see how it’s encoding DMs.

[Q_Test_1]

Hi,

Thank you for your response. I should have mentioned that my OAuth signature is always calculated based upon whatever the parameter values are. So, if I update my code to change the encoding of parameter(s) then the new values are used to calculate the OAuth signature, timestamp etc.

I will dig into twitter4j now and see what I can find. Thanks for the tip!

[jublonet]

I will dig into twitter4j now and see what I can find.

Can you please update us with your findings?

[Q_Test_1]

I have resolved the issue, thanks to a hint from abraham.

On Android, URLEncoder class exists which encodes a string to HTML form standards. For URL parameters, we must replace “+” with “%20” in the return value from the URLEncoder.encode() method.

I as doing the replace when encoding the parameter before the request was sent, but I was not doing the replace when calculating the OAuth signature (URL parameters form part of the OAuth signature). So, I guess on Twitter’s side, the decompiled OAuth signature parameters did not fully match the URL parameters, hence the 401 (authentication error).

[jublonet]

Just to compare — this is the code used in Codebird for escaping:

PHP

  /**
   * URL-encodes the given data
   *
   * @param mixed $data
   *
   * @return mixed The encoded data
   */
  protected function _url($data)
  {
    if (is_array($data)) {
      return array_map([
        $this,
        '_url'
      ], $data);
    } elseif (is_scalar($data)) {
      return str_replace([
        '+',
        '!',
        '*',
        "'",
        '(',
        ')'
      ], [
        ' ',
        '%21',
        '%2A',
        '%27',
        '%28',
        '%29'
      ], rawurlencode($data));
    }
    return '';
  }

---

JavaScript

/**
 * URL-encodes the given data
 *
 * @param mixed data
 *
 * @return mixed The encoded data
 */
_url(data) {
  if ((/boolean|number|string/).test(typeof data)) {
    return encodeURIComponent(data).replace(/!/g, "%21").replace(/'/g, "%27").replace(/\(/g, "%28").replace(/\)/g, "%29").replace(/\*/g, "%2A");
  } else {
    return "";
  }
}