So i am building a gamified engagement system in which people will be logging into my app in a timeframe of say 3-4 hours. And the number of users can range from 4000-5000. I want to track each users activity in twitter during the time frame. I will be getting data like tweets,retweets,favourites etc… Since i need to track the activity outside the twitter app, i will be recording the oauth - token of the user .
This is how i am planning to do this
Set up one cron job that run through out the time frame fetch what each user’s activity is using respective oauth token and update it in the main app. Also i can make sure in a 15 minute time only max of 100 requests are done from each user token.
I would like to know how efficient is this. Or please suggest me a better way to handle this.