Search/tweets.json returns only up to the recent 2500-3000 tweets from today


#1

i am making this application where i’m doing simple tweet-counting with a specific keyword. i have read in the specification that the tweets.json returns only tweets for about a week back, which was fine for me.

so, i have implemented a simple recursive search that would keep getting tweets with lower and lower max_ids (using the search_metadata->next_results) until there are either none to fetch or i hit my time limit. the script runs good on unpopular tweets. however, now i’ve tried to run the script on actually popular hashtags (for instance “#awesome”) and it returns about 3000 latest tweets from today only (the number is pretty random too. sometimes i get 100, sometimes 2000, sometimes 3000, once i got 7000). no “archive of week-old tweets” like the specification says. it is very chaotic

is there a limit on maximum of stored tweets for a specific keyword too? do i have to constantly run the script every hour and store the latest values in my own database? some kind of local cache?

i’d really like some clarification on this, as i haven’t been able to find information any imposed message limit restrictions.
please help