Streaming API for a lot of users?


I have been using the Twitter RESTful API for quite a while now. I’m starting to get rate limited more and more and I am looking to migrate into using the streaming API.

I am wondering what the stepping stones should be. The way I work with the API is that I poll certain “shares” made from my website for about a week once an hour. I think that the “follow” stream would potentially work well but my research throughout the dev forums shows that it allows up to a maximum of 5,000 followers. We have about 400,000 people on our site, however. Albeit, a lesser portion are active, but I definitely have more than 5,000 active users.

It seems to me that site stream is the way to go for us. However this requires certain preliminary steps from what I’ve read such as implementing a user stream successfully, except the problem is that the user stream doesn’t seem like what I want.

I have scoured the forums for a while and I can’t figure out any up-to-date straightforward guides.