Twitter4J compatibility issue with Control Streams discovered

[LogicalArthur]

Thanks to reports from several developers, we have discovered that with a recent backend change, [node:2835] now misbehaves on Twitter4J.

Before deploying the change, our engineering teams did extensive testing against raw cURLs and our homegrown Hosebird Client (https://github.com/twitter/hbc) and we found no problems when conforming to established HTTP standards.

While we cannot conclude with 100% certainty at this time that the error lives with the Twitter4J library, we advise you to check your applications and code to make sure that you are not affected.

Some developers have reported that a temporary workaround is disabling gzip compression on Control Streams requests.

If you have anything to share on this matter, including any inquiries, please use this discussion thread.

Thank you!

[aaronrankin]

I can +1 that we experienced this at Sprout Social and we worked around it by disabling GZIP.

[yusuke]

Hello,

I’m currently disabling unit tests on my Jenkins instance as my test application doesn’t have read-write-directmessages access level.
I created a new app with read-write-directmessages access level and am waiting for approval from @twitterapi.

t4j_news (t4j_news)

Until the request got approved, I can’t run unit tests against site streams.

Thanks,
Yusuke

[graysky]

@yusuke - thanks for looking into this!

I posted about the errors we saw in the twitter4j mailing list:
https://groups.google.com/forum/?hl=en#!topic/twitter4j/n9pmOslf_B8

[nosuz]

Hi,

I am thinking this issue is not only on the Twitter4J because I had the same problem on my Application written by Ruby.

My work around is to change the inflater method for HTTP response body.

The version of ruby is 1.9.3p0 (2011-10-30 revision 33570) [x86_64-linux]

Before:
response_body = Zlib::GzipReader.wrap(StringIO.new(response.body)).read

After:
zstream = Zlib::Inflate.new(Zlib::MAX_WBITS + 32)
response_body = zstream.inflate(response.body)

I am not sure what is the difference on them but it works.

Thank you.

[james_jory]

We experienced this issue as well (using HBC + twitter4j). Worked around by disabling gzip.

[ltucker]

I see this in python as well.

@nosuz MAX_WBITS + 32 means use zlib header detection. This may indicate certain “gzip” (rfc 1952) streams are in fact something else like zlib or deflate (rfc1950, rfc1951) …

@LogicalArthur clients like curl may not be a great test of compliance because they contain fall-back behavior that works around improperly reported compression formats. In particular curl will fall back to looking for zlib headers. eg https://github.com/bagder/curl/blob/master/lib/content_encoding.c#L146