Keyword based search within the historical tweets of a particular user

[parangsaraf]

Hi,

I have a list of users (say 10 users). I also have a list of keywords (say 2 keywords: happy and sad).

Now, I want to search for all the tweets in the history of these users mentioning either “happy” or “sad”.

I know of one way of doing this:

1. For every user using the user_timeline get their last 3200 tweets ( rate is limited to 180 requests per window. So around 18 windows)
2. Then from that collection search for the tweets which mention “happy” or “sad”.

Now, this looks like a really really bad way of solving this problem to me. To make it worse, assume that for a particular user, in the end, we found only 5 relevant tweets (that mentioned “happy” or “sad”). So, fetched 3200 tweets, wasted lot of time waiting on windows, just to fetch 5 relevant tweets. Now multiply this for 2000 users and 100 keywords.

Other Restriction: I want to use only twitter api and no third party solutions (reliability issues).

Any Suggestions?? I will greatly appreciate your help.

Regards

[episod]

The strategy you detailed is currently the best strategy to use for this purpose – we don’t offer a focused search against a group of users in the API at this time.

[parangsaraf]

I just hope twitter doesn’t blacklist me. Thanks for the reply.

[episod]

If you follow the rate limits, respond to HTTP codes correctly, and are otherwise following the TOS you shouldn’t get “blacklisted.”