Using 'next' token parameter for data pagination of POST request

[psoon6]

Hi,

I am collecting historical twitter data using sandbox full-archive endpoint.
I have generated a code that search tweets with specific keywords with max results of 100. My results has shown that there are more results to be returned as identified by the ‘next’ token in the output. I am following the steps from this link https://developer.twitter.com/en/docs/tweets/search/api-reference/premium-search to request for more results using ‘next’ parameter with the value used with the parameter pulled directly from the response provided by the API . However it is not working and I get an error message in my results saying “Invalid json, could not parse.”

Second question: For sandbox full-archive, there is maximum request of 50 per month with 100 results maximum then does that mean that if I am requesting very large amounts of tweets then I may go over my request limits before I reach the end of my results (using ‘next’ token)?

Thanks!

[happycamper]

Hi @psoon6 - When using the next token, be sure to specify all the original parameters along with the next token in the json input. Also if you are using the GET (vs. POST) be sure to encode your next token.

For your second question- the initial request, as well as all next token/pagination requests count toward the request limit (50 per month).