Hbc. Streaming API low rate


#1

I’m using hbc to read from the twitter streaming API following using pretty much unmodified hbc-example/src/main/java/com/twitter/hbc/example/SampleStreamExample.java.

If I suppress the input and just print out some stats:
The client read 5000 messages, 17 Mbytes in 99s (0.18 MB/s)

After tweaking some parameters here and there, remains pretty constant. Both my local box and a cloud deployment show very similar numbers.

I’ve checked all stats in Wireshark and the rate is constant, there are no disconnects, or pause due to rate limiter. Any hints? I assume the rate should be as much as my box can handle.


#2

50 per second is about right. Your assumption is wrong. Sample stream docs.


#3

Thanks. I had vastly overestimated how large this 1% would be.
I also found some numbers in the twitter blog that confirm your statement.


#4