I was getting 3000+ tweets per minute, that has decreased slowly to now about 100 per minute. Could someone explain why please?

MichaelCouck · 2014-01-17T14:27:48.000Z

Hello,

Slowly over the last several months, my tweet rate has been decreasing, from 3000++ per minute to around 100 now, which is not usable anymore. This means that I have to change the source of data because the volume is just not enough to train classifiers anymore. Potentially a week or two of development. I would rather have my rate back up to 3000, as all the code is tested and released already. Failing that I have to look at the other social networks for source data.

Thanks in advance,
Michael

MichaelCouck · 2014-01-25T12:49:49.000Z

Well magically I am back up to 3000 tweets a minute Hmmm… very strange.

doublehelix · 2014-01-28T14:48:53.000Z

I am having a similar issue - the rate dropped dramatically (by a factor of 5, looking at my bandwidth readings, not actual tweet counts) on 15 Jan 2014.

This is using the Sample Stream.

I am using Twitter4j v3.0.5.

No code or config changes were made over this period at my end.

Perhaps compression has been enabled on the stream, which wasn’t previously enabled…? (Given that I have been looking at bandwidth not tweet counts…).

UPDATE: the overall number of tweets received per day is approximately the same as before the change, so I can only conclude that compression has suddenly started working (which is nice)!