I have been working with the streaming API for some time now and now that I know how it works, I would like to take it to the next level where it can be used in production environments. I have a few questions/ remarks regarding the same which I would like to share and get feedback on before continuing:
I made the first program using curl and Perl. Does any of you have experience with that? I would like to know if it is a decent choice to also put retry and fail, monitor mechanism around the code or should I look for a different language(python?) or use sockets with net api ? The tweets are then stored in a db to be consumed by the front end app
I am planning to also have a rest API which can request missing tweets using the since and max_id parameters to grab the missing tweets in case of a restart.
The app will track certain keywords which might change over time, now when a new keyword is added or an old one is removed, how can I handle that?
one way is to close the stream and open a new one with the new list, the other will be to open another stream on the same account and let twitter close this stream automatically avoiding any tweet loss. The third option will be to open a new stream with different credentials and then close the old stream. The change in the parameters will be happening in the worst case 1-2 times a day. (normally once per week)
Thanks for your replies in advance!