Scaling Streaming Connection


#1

Hi,

I am using the twitter streaming api to track certain keywords and I use php phirehose and everything works great.

Now, my question is - currently, I track @mentions or certain keywords ( like company names ) which don’t get huge traffic per sec and I am able to manage the load.

But, now I want to give my user an option to track just about anything.

eg #love.

This definitely will get our server down as streaming connection is opened from only one box.

I am worried as to how we can split this load ? We have a setup with LB and many servers but how can I open this connection so load is spread across servers ? Or how powerful should a box be to handle tracking something like #love ?

Let me know.
Thanks.