Search/tweets.json endpoint giving faulty paging tokens


#1

I am using the search/tweets.json endpoint to retrieve the previous days tweets that I may have missed using the twitter streaming API.

In order to fully do this, I need to incorporate the pagination that twitter does in case my results go over the designated number per page.

Here is the result that I receive from the Twitter debug console on the Twitter for Mac app:

"search_metadata": { 
    "completed_in": 0.012, 
    "max_id": 110518036596084740, 
    "max_id_str": "110518036596084735", 
    "next_results": "?max_id=110488276361240576&q=%23love%20since%3A2015-01-28&count=2&include_entities=1", 
    "query": "%23love+since%3A2015-01-28", 
    "refresh_url": "?since_id=560518036596084735&q=%23love%20since%3A2015-01-28&include_entities=1" etc, etc.

The important part is the ‘next_results’. This is completely inconsistent. This is (I assume) supposed to be a parameter string that you can use the get the next page, right? For some reason there are also random characters inserted into the string. The %20, or whitespace character is inserted before the ‘since’ parameter, which for SOME reason isn’t using an ‘=’ sign, but instead a colon ( ‘:’ ). It seems like the whitespace character replaced the ‘&’ in the query - but i don’t know why, and I don’t like it.

Not only that, the max_id element in that JSON string isn’t even the max_id for the next page, it literally doesn’t help you AT ALL. I have to go in, grab the last element of the tweet array, grab that tweet id and then decrement it by 1, AND THEN use it in the max_id.


#2

In my case paging is returning the same 20 results.