Keyword based search within the historical tweets of a particular user



I have a list of users (say 10 users). I also have a list of keywords (say 2 keywords: happy and sad).

Now, I want to search for all the tweets in the history of these users mentioning either “happy” or “sad”.

I know of one way of doing this:

  1. For every user using the user_timeline get their last 3200 tweets ( rate is limited to 180 requests per window. So around 18 windows)
  2. Then from that collection search for the tweets which mention “happy” or “sad”.

Now, this looks like a really really bad way of solving this problem to me. To make it worse, assume that for a particular user, in the end, we found only 5 relevant tweets (that mentioned “happy” or “sad”). So, fetched 3200 tweets, wasted lot of time waiting on windows, just to fetch 5 relevant tweets. Now multiply this for 2000 users and 100 keywords. :open_mouth: :open_mouth:

Other Restriction: I want to use only twitter api and no third party solutions (reliability issues).

Any Suggestions?? I will greatly appreciate your help.



The strategy you detailed is currently the best strategy to use for this purpose – we don’t offer a focused search against a group of users in the API at this time.


I just hope twitter doesn’t blacklist me. Thanks for the reply.


If you follow the rate limits, respond to HTTP codes correctly, and are otherwise following the TOS you shouldn’t get “blacklisted.”