MaxId Parameter Ignored?


#1

Last night I started using the search/tweets endpoint to request tweets matching a certain query. I used the approach described here https://dev.twitter.com/docs/working-with-timelines to collect older and older tweets, and everything worked as expected. Then this morning I killed the process to make some unrelated changes, and restarted the process with the initial MaxId set to the earliest ID I had collected. I expected that the queries would pick up where they left off (November 2, 2013) and continue going back in time. However, I instead started receiving tweets from today (November 10, 2013). Once again the queries behaved correctly from that point on, moving back gradually from the November 10 start point, but this will cause a lot of duplicate work once it reaches the tweets that I already collected last night. I also tried adding until=2013-11-02 to my query, but when I did that the query seemed to hang and I didn’t receive any response.

Is there some limit on how far back you can go with the search/tweets query? I tried using IDs from a November 7 tweet and a November 4 tweet as my initial max ID and each of those worked as expected, but IDs from tweets on November 2 don’t seem to work. It’s also strange to me that the initial process ran without problems for November 2 until I manually killed it, but then as soon as I restarted the process a minute or so later, I encountered this problem.

Thanks in advance for any help or insight.


#2

I suspect now that there is a cutoff of about one week-- I did some experimenting with different MaxId values and I found a cutoff point with a date of about one week ago such that all IDs after that point behave as expected, and all IDs before revert to the present date.


#3

Yes, search results from the search/tweets API generally only go back in time about one week – sometimes less but seldom longer.