I’m working on a project for an NLP course I’m taking and I’d like to be able to get larger amounts of data to make the application more robust. I’ve run into two issues:
- Multiple searches return the same list of tweets so doing multiple searches for 100 tweets doesn’t actually give me more than 100 tweets
- I’ll most likely hit the rate limit pretty quickly if I’m grabbing searches from different topics.
Are there any recommendations for either issue? Would it make sense to use the Streaming API if I’ll be switching hashtags a lot? I was also thinking that it may make more sense to find just archived tweets and parse that instead of using the API.