Streaming: "Transfer-Encoding" header sometimes lower-case ("transfer-encoding: chunked")


#1

Relatively rarely, connecting to the Streaming API at https://stream.twitter.com/1.1/statuses/filter returns the “Transfer-Encoding” header in lower-case: “transfer-encoding: chunked”. My understanding is that it should always be capitalized, and while I can adjust my client, some libraries may choke on this. Ref: http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-26#section-3.3.1

Here’s a log of all headers that were sent in such a case:

connection: close
Content-Type: application/json
transfer-encoding: chunked
date: Tue, 22 Apr 2014 17:14:08 UTC
x-connection-hash: 17b9b9d305a66496fc4e703e8f3d8ee9

Thanks,
Phil


#2

We are also experiencing this.


#3

actually we got lowercase header: transfer-encoding: chunked


#4

It appears as if the lower-case version is now returned on every connect. I would have to speculate that there was a release today that introduced this behaviour.


#5

So that would explain why the phirehose library is getting an error:

PHP Fatal error: Uncaught exception ‘Exception’ with message ‘Twitter did not send a chunking header. Is this really HTTP/1.1? Here are headers:
connection: close
Content-Type: application/json
transfer-encoding: chunked
date: Wed, 23 Apr 2014 00:52:11 UTC
x-connection-hash: e429d181bd6b89f18d185c81fe83d3a4
’ in /var/www/html/SE/140dev/libraries/phirehose/Phirehose.php:739
Stack trace:
#0 /var/www/html/SE/140dev/libraries/phirehose/Phirehose.php(830): Phirehose->connect()
#1 /var/www/html/SE/140dev/libraries/phirehose/Phirehose.php(409): Phirehose->reconnect()
#2 /var/www/html/SE/140dev/db/get_tweets.php(263): Phirehose->consume()
#3 {main}
thrown in /var/www/html/SE/140dev/libraries/phirehose/Phirehose.php on line 739


#6

Yes, I pushed a fix for this to Phirehose today and https://github.com/DarrenCook already merged into master (thanks!). See https://github.com/fennb/phirehose/pull/63 and https://groups.google.com/forum/#!topic/phirehose-users/AytivyXTxrw


#7

Thanks for that, btw.


#8

I’ve just had exactly the same issue and this is now fixed thanks to your fix on Phirehose.

Thanks :slight_smile:


#9

Thank you for the info and the fast fix.


#10

Thanks for the quick fix!


#11

Thanks @hootphil we needed this quick fix for stunx.com!


#12

Tks tks tks!!! Thanks to your fix on Phirehose.


#13

You’re awesome.


#14

Lol my phirehose is not recognisable as phirehose anymore. I will have to adjust manually had it fixed in 2 minutes though but now I can apply a final solution, thanks devs.


#15

Hey everyone. Thanks for your patience while we investigated this issue.

Take a look at http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2

In particular, note “Field names are case-insensitive.”.

We’re very happy to hear that Phirehose was patched, and we encourage others to update their clients and libraries to adhere to the standard and not hard-code a particular capitalization for header field names.


#16

Thanks for looking into this Arthur. After checking the specs in more detail, I realized that the document I previously referred to states this case-insensitivity as well (http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-26#section-3.2).

Good to get clarification on this, and at least this discussion will serve as a reference for other developers.

Have a good weekend,
Phil