#1

When you paginate back inside the first 100 retweets, for example with a max_results of 25, everything seems to be working as expected (with a strange offset, I’ll talk about it in the example bellow). But as soon as you try to paginate back from the 101st+ entries, you will always get back somewhere inside the latest 100 entries.

I’m sorry if my explanation is not clear enough, I’ll do my best to demonstrate it in the following example:
We need a tweet having 200 public retweets, let’s use this one published by @/Twitter, having 266 RTs.
When I load the first page by putting no pagination_token, I get this data (I’ve made sure to strip enough data for privacy concerns):

/* First page (https://api.twitter.com/2/tweets/1485659634866266114/retweeted_by?max_results=25 */
{
  "data": [
      {
          "id": "...8414824452"
      },
      {
          "id": "...5631291392"
      },
      {
          "id": "...44986"
      },
/* 19 more results */
      {
          "id": "...1855"
      },
      {
          "id": "...020376577"
      },
      {
          "id": "...3169843200"
      }
  ],
  "meta": {
      "result_count": 25,
      "next_token": "7140dibdnow9c7btw480xr26dekrkrscx6wloiimx6ng9"
  }
}

/* Page 2, using next_token from page 1 (https://api.twitter.com/2/tweets/1485659634866266114/retweeted_by?max_results=25&pagination_token=7140dibdnow9c7btw480xr26dekrkrscx6wloiimx6ng9) */
{
  "data": [
      {
          "id": "...14779"
      },
      {
          "id": "...94053"
      },
      {
          "id": "...91389"
      },
/* 19 more results */
      {
          "id": "...3662437379"
      },
      {
          "id": "...8524780548"
      },
      {
          "id": "...69855"
      }
  ],
  "meta": {
      "result_count": 25,
      "next_token": "7140dibdnow9c7btw480xr26bvqpas7216qyzxfoktkj8",
      "previous_token": "77qpymm88g5h9vqkluxce6kj4eafg4novob3be656mypd"
  }
}

/* Page 3, using next_token from page 2 (https://api.twitter.com/2/tweets/1485659634866266114/retweeted_by?max_results=25&pagination_token=7140dibdnow9c7btw480xr26bvqpas7216qyzxfoktkj8) */
{
  "data": [
      {
          "id": "...04906"
      },
      {
          "id": "...5907208192"
      },
      {
          "id": "...2557298690"
      },
/* 19 more results */
      {
          "id": "...00043"
      },
      {
          "id": "...9173790720"
      },
      {
          "id": "...8389948418"
      }
  ],
  "meta": {
      "result_count": 25,
      "next_token": "7140dibdnow9c7btw480xr26bu8yosbdhew3xgeeawe2d",
      "previous_token": "77qpymm88g5h9vqkluxce6kj2vnt2m4j0az7cjxp2o1fd"
  }
}

Now, since we are at page 3, let’s look at what we get when go back to page 2 by using previous_token from page 3 (77qpymm88g5h9vqkluxce6kj2vnt2m4j0az7cjxp2o1fd)

/* This is supposed to be page 2, using previous_token from page 3 (https://api.twitter.com/2/tweets/1485659634866266114/retweeted_by?max_results=25&pagination_token=77qpymm88g5h9vqkluxce6kj2vnt2m4j0az7cjxp2o1fd) */
{
  "data": [
      {
          "id": "...57878"
      },
      {
          "id": "...2730956801"
      },
      {
          "id": "...1855"
      },
/* 19 more results */
      {
          "id": "...0877468673"
      },
      {
          "id": "...9391245312"
      },
      {
          "id": "...5011335169"
      }
  ],
  "meta": {
      "result_count": 25,
      "next_token": "7140dibdnow9c7btw480xr26dcnfa29it6wi06qmmo2ew",
      "previous_token": "77qpymm88g5h9vqkluxce6kj5w9ts73gnnn1tz2pzd2d4"
  }
}

We can already see that the entries don’t correspond to the 3 first & 3 last ids we got on page 2, there is a slight offset of 5 entries, you can see in a max_results=100 output that those entries are located between indexes 20 to 44 (both inclusive, start-index at 0), but they should be 25 to 49.
If you want to have a look at the same output I got when requesting the last 100 retweeters, I’ve uploaded a copy here.

Let’s get page 4 (by using next_token returned in page 3 output) then page 5…

/* Page 4, using next_token from page 3 (https://api.twitter.com/2/tweets/1485659634866266114/retweeted_by?max_results=25&pagination_token=7140dibdnow9c7btw480xr26bu8yosbdhew3xgeeawe2d) */
{
  "data": [
      {
          "id": "...9308969985"
      },
      {
          "id": "...8050660355"
      },
      {
          "id": "...13342"
      },
/* 19 more results */
      {
          "id": "...9893370880"
      },
      {
          "id": "...9832250372"
      },
      {
          "id": "...107908608"
      }
  ],
  "meta": {
      "result_count": 25,
      "next_token": "7140dibdnow9c7btw480xr26bu12kv30m40da789nhwdt",
      "previous_token": "77qpymm88g5h9vqkluxce6kj2udxgwky0w0i3xrfkbcj7"
  }
}

Page 5:

/* Page 5, by using next_token from page 4 (https://api.twitter.com/2/tweets/1485659634866266114/retweeted_by?max_results=25&pagination_token=7140dibdnow9c7btw480xr26bu12kv30m40da789nhwdt) */
{
  "data": [
      {
          "id": "...6697100288"
      },
      {
          "id": "...38272"
      },
      {
          "id": "...4234686465"
      },
/* 19 more results */
      {
          "id": "...8562197508"
      },
      {
          "id": "...3755014144"
      },
      {
          "id": "...0969804801"
      }
  ],
  "meta": {
      "result_count": 25,
      "next_token": "7140dibdnow9c7btw480xr26adbqbtpi9vvvou9lq62pi",
      "previous_token": "77qpymm88g5h9vqkluxce6kj2u60auipujg66pifssy7n"
  }
}

And page 6:

/* This is page 6, we fetch it only to get the previous_token that is supposed to get us back to page 5 (https://api.twitter.com/2/tweets/1485659634866266114/retweeted_by?max_results=25&pagination_token=7140dibdnow9c7btw480xr26adbqbtpi9vvvou9lq62pi) */
{
  "data": [
      {
          "id": "...6228784128"
      },
      {
          "id": "...21145"
      },
      {
          "id": "...49738"
      },
/* 19 more results */
      {
          "id": "...67032"
      },
      {
          "id": "...4823356416"
      },
      {
          "id": "...0665470977"
      }
  ],
  "meta": {
      "result_count": 25,
      "next_token": "7140dibdnow9c7btw480xr26ad48bebw2vjtmu32l5naq",
      "previous_token": "77qpymm88g5h9vqkluxce6kj1dgq67w1ld99d4rzkhkc0"
  }
}

Now let’s go back to page 5 (by using previous_token from page 6)

/* This is supposed to be page 5, by using previous_token from page 6 (https://api.twitter.com/2/tweets/1485659634866266114/retweeted_by?max_results=25&pagination_token=77qpymm88g5h9vqkluxce6kj1dgq67w1ld99d4rzkhkc0) */
{
    "data": [
        {
            "id": "...57878"
        },
        {
            "id": "...2730956801"
        },
        {
            "id": "...1855"
        },
/* 19 more results */
        {
            "id": "...0877468673"
        },
        {
            "id": "...9391245312"
        },
        {
            "id": "...5011335169"
        }
    ],
    "meta": {
        "result_count": 25,
        "next_token": "7140dibdnow9c7btw480xr26dcnfa29it6wi06qmmo2ew",
        "previous_token": "77qpymm88g5h9vqkluxce6kj5w9ts73gnnn1tz2pzd2d4"
    }
}

As you can see, we get the exact same output as when we tried to go from page 3 to page 2, with the same weird offset of 5 entries. That’s really far from being the entries 100th-ish to 125th-ish entries we should get.

So it seems that using previous_token doesn’t give reliable results. I hope you won’t struggle too much to reproduce the case I just exposed, and to find (and hopefully fix) the origin of this issue.
I’m happy to give any more details or explanations if necessary.
Thanks

3 Likes
#2

Hi @richie3366,

I was able to recreate your case. Unfortunately I don’t have a solution, but thought it might encourage some discussion if someone else also saw this.

At first I thought it might had been an issue with that tweet getting more re-tweets while the API request for re-tweets was being made, but that was not the case. I tried on an older tweet, 1468347983838134274, that was not seeing as much activity and got a similar result of unexpected behavior.

My script in a loop of 10 made a call with current next_token, saved those results, made a call with the previous_token and compared those results to the results of the previous iteration of the loop’s api call with the next_token. Thinking the last iteration values should be the same as a call with ‘previous_token’ in this iteration. That didn’t work.

Second thought was that there was something with the token being invalidated if the APIs calls ‘changed direction’, so I tried

  • 10 calls “forward” paginating with the ‘next_token’ value, 0 to 9
  • grab the most recent ‘previous_token’ value
  • then make 9 calls “backward” paginating with the ‘previous_token’, 8 to 0

(the ‘direction pivot’ is on page 9. numbering is just for keeping tack of expected results, not used in the api calls)

The idea being the first set of results and the last set of results should be the same. In other words “count up to 10 from 0, then count down from 10 and expect to end at 0”.

This broke in a very weird way where the api only returned 2 “backwards” results before not including a ‘previous_token’.

And the results were not what I would expect in pagination.

I would have expected “forward page 8” and “backward page 8” to be the same. But “backward page 8” had results of “forward page 0” with some offset, “backward page 7” had results of “forward page 0” with NO offset and that was it.

The output of the 10-forward-10-backward is below. For readability I tweaked the response JSON for the ‘data’ key to just be the ID values, instead of the id, name, username dict. I kept the rest of the output, like the token values, intact incase someone could spot a flaw in my pagination that would explain the behavior.

Anyway, hope you get some feedback that helps resolve this seemingly repeatable issue.

Best of luck,
rdp

2022-01-28 18:32:36 DEBUG FORWARD PAGE 0
2022-01-28 18:32:36 DEBUG https://api.twitter.com:443 “GET /2/tweets/1468347983838134274/retweeted_by?max_results=25 HTTP/1.1” 200 891
2022-01-28 18:32:36 DEBUG {‘data’: [‘1105855292443713537’, ‘245634496’, ‘24807828’, ‘3130347809’, ‘10680442’, ‘981519459306307584’, ‘1233080592’, ‘8657052’, ‘455900925’, ‘364499002’, ‘7153662’, ‘2645987132’, ‘234760327’, ‘203219181’, ‘755352’, ‘539458560’, ‘3769484542’, ‘426039231’, ‘765941712351027200’, ‘14228625’, ‘715598587816620032’, ‘19612717’, ‘49436627’, ‘2543762662’, ‘44961598’], ‘meta’: {‘result_count’: 25, ‘next_token’: ‘7140dibdnow9c7btw480j0b81ofbt02pm0qm9kwlwj5f6’}}
2022-01-28 18:32:36 DEBUG FORWARD PAGE 1
2022-01-28 18:32:36 DEBUG https://api.twitter.com:443 “GET /2/tweets/1468347983838134274/retweeted_by?max_results=25&pagination_token=7140dibdnow9c7btw480j0b81ofbt02pm0qm9kwlwj5f6 HTTP/1.1” 200 940
2022-01-28 18:32:36 DEBUG {‘data’: [‘1253694008611573760’, ‘20698952’, ‘489989624’, ‘15218327’, ‘372081433’, ‘14326197’, ‘1106227004024725505’, ‘198629740’, ‘23600888’, ‘872144505172094976’, ‘17727527’, ‘18749899’, ‘1343610488081379328’, ‘384198613’, ‘14298593’, ‘1408533062’, ‘732041910877978624’, ‘571543394’, ‘108101617’, ‘17927953’, ‘1461625477’, ‘275843300’, ‘1633239618’, ‘3182034907’, ‘14354574’], ‘meta’: {‘result_count’: 25, ‘next_token’: ‘7140dibdnow9c7btw480j0b806g1shcfl2zd80somlfsp’, ‘previous_token’: ‘77qpymm88g5h9vqkluxbzftksoctn4h35rep15w19t0dx’}}
2022-01-28 18:32:36 DEBUG FORWARD PAGE 2
2022-01-28 18:32:37 DEBUG https://api.twitter.com:443 “GET /2/tweets/1468347983838134274/retweeted_by?max_results=25&pagination_token=7140dibdnow9c7btw480j0b806g1shcfl2zd80somlfsp HTTP/1.1” 200 1079
2022-01-28 18:32:37 DEBUG {‘data’: [‘1251617479484493825’, ‘355318675’, ‘805054484’, ‘3184826318’, ‘216058226’, ‘16783281’, ‘2445387283’, ‘1928268895’, ‘850201293003948034’, ‘292068601’, ‘35600300’, ‘18150270’, ‘1333538552496476163’, ‘60267638’, ‘909596695020916737’, ‘914668145801142272’, ‘5749002’, ‘6479502’, ‘17332866’, ‘1575912229’, ‘466611416’, ‘9644882’, ‘1220534589145223169’, ‘52885052’, ‘934627337701920768’], ‘meta’: {‘result_count’: 25, ‘next_token’: ‘7140dibdnow9c7btw480j0b8060qxe6pym59ilupi8304’, ‘previous_token’: ‘77qpymm88g5h9vqkluxbzftkr6l1n0y9yxjax8grhpl4g’}}
2022-01-28 18:32:37 DEBUG FORWARD PAGE 3
2022-01-28 18:32:37 DEBUG https://api.twitter.com:443 “GET /2/tweets/1468347983838134274/retweeted_by?max_results=25&pagination_token=7140dibdnow9c7btw480j0b8060qxe6pym59ilupi8304 HTTP/1.1” 200 1044
2022-01-28 18:32:37 DEBUG {‘data’: [‘8814212’, ‘540640674’, ‘2660241685’, ‘11604242’, ‘2316095946’, ‘437984839’, ‘14414720’, ‘1137715867’, ‘37619775’, ‘376298012’, ‘3107483711’, ‘349208036’, ‘44504483’, ‘17098686’, ‘3216164855’, ‘84987375’, ‘112631234’, ‘1555221686’, ‘27152576’, ‘23278923’, ‘336443569’, ‘65688592’, ‘3478338192’, ‘312384480’, ‘285150493’], ‘meta’: {‘result_count’: 25, ‘next_token’: ‘7140dibdnow9c7btw480j0b8060lj44yclho12jrkdjqq’, ‘previous_token’: ‘77qpymm88g5h9vqkluxbzftkr65polmdveyf4pgv0o0mx’}}
2022-01-28 18:32:37 DEBUG FORWARD PAGE 4
2022-01-28 18:32:37 DEBUG https://api.twitter.com:443 “GET /2/tweets/1468347983838134274/retweeted_by?max_results=25&pagination_token=7140dibdnow9c7btw480j0b8060lj44yclho12jrkdjqq HTTP/1.1” 200 951
2022-01-28 18:32:37 DEBUG {‘data’: [‘353419774’, ‘18849368’, ‘1017586704935464960’, ‘451189887’, ‘208422641’, ‘13104972’, ‘22775788’, ‘16793718’, ‘1227844757813989376’, ‘208086641’, ‘1247736358409351170’, ‘1139046187’, ‘40434410’, ‘45323315’, ‘29837794’, ‘60378902’, ‘1055028384323911682’, ‘65027669’, ‘1202875621’, ‘96602143’, ‘1543761’, ‘2448231’, ‘43503’, ‘696163’, ‘111579405’], ‘meta’: {‘result_count’: 25, ‘next_token’: ‘7140dibdnow9c7btw480iy8adszdnjr7svsdymk1g592v’, ‘previous_token’: ‘77qpymm88g5h9vqkluxbzftkr65ld7hl2mzp2xz434oj7’}}
2022-01-28 18:32:37 DEBUG FORWARD PAGE 5
2022-01-28 18:32:38 DEBUG https://api.twitter.com:443 “GET /2/tweets/1468347983838134274/retweeted_by?max_results=25&pagination_token=7140dibdnow9c7btw480iy8adszdnjr7svsdymk1g592v HTTP/1.1” 200 1002
2022-01-28 18:32:38 DEBUG {‘data’: [‘65399345’, ‘92220058’, ‘485507437’, ‘14723614’, ‘1788441’, ‘31740492’, ‘2657944608’, ‘36551220’, ‘880183289516097536’, ‘56446996’, ‘48003589’, ‘981591766502690818’, ‘21496683’, ‘756197006171779072’, ‘833373698’, ‘377151387’, ‘3450800300’, ‘249361066’, ‘902735146037649408’, ‘49052724’, ‘14603117’, ‘1068443406’, ‘88764989’, ‘788092405’, ‘1468803615544520704’], ‘meta’: {‘result_count’: 25, ‘next_token’: ‘7140dibdnow9c7btw480iy7zmwvgkk3mnxz76rjt15pgx’, ‘previous_token’: ‘77qpymm88g5h9vqkluxbzdqn3bs1tcaccc32zy70kx9h4’}}
2022-01-28 18:32:38 DEBUG FORWARD PAGE 6
2022-01-28 18:32:38 DEBUG https://api.twitter.com:443 “GET /2/tweets/1468347983838134274/retweeted_by?max_results=25&pagination_token=7140dibdnow9c7btw480iy7zmwvgkk3mnxz76rjt15pgx HTTP/1.1” 200 1004
2022-01-28 18:32:38 DEBUG {‘data’: [‘17125397’, ‘1101507945202888706’, ‘791833176314417152’, ‘538512792’, ‘1439117031320735744’, ‘366067504’, ‘1203495444132827136’, ‘109975086’, ‘1085023245718437889’, ‘1109974599012438018’, ‘3313848900’, ‘1445233599263899648’, ‘3565687879’, ‘1010077316’, ‘96015170’, ‘14815682’, ‘27647185’, ‘18051104’, ‘86348953’, ‘14584235’, ‘362089000’, ‘773241147732979712’, ‘242078036’, ‘5614032’, ‘17977180’], ‘meta’: {‘result_count’: 25, ‘next_token’: ‘7140dibdnow9c7btw480iy7zjuzh0avn1xcdxn3hz7iuu’, ‘previous_token’: ‘77qpymm88g5h9vqkluxbzdqcdx0fda750djr7cvbk0vue’}}
2022-01-28 18:32:38 DEBUG FORWARD PAGE 7
2022-01-28 18:32:38 DEBUG https://api.twitter.com:443 “GET /2/tweets/1468347983838134274/retweeted_by?max_results=25&pagination_token=7140dibdnow9c7btw480iy7zjuzh0avn1xcdxn3hz7iuu HTTP/1.1” 200 921
2022-01-28 18:32:38 DEBUG {‘data’: [‘19234236’, ‘197978371’, ‘14432742’, ‘15453583’, ‘247630288’, ‘1119385290731397121’, ‘527884521’, ‘35498483’, ‘2244305374’, ‘458610932’, ‘844705818’, ‘11614102’, ‘114290226’, ‘15559361’, ‘2411730192’, ‘1092936374624370688’, ‘932101266687365121’, ‘173212823’, ‘623354495’, ‘1158665890172461056’, ‘19837512’, ‘1876932115’, ‘166416786’, ‘9035752’, ‘1462560314625458177’], ‘meta’: {‘result_count’: 25, ‘next_token’: ‘7140dibdnow9c7btw480iy7ziduqm5c5ytdlnxboa3msx’, ‘previous_token’: ‘77qpymm88g5h9vqkluxbzdqcav4fskfytofgfgn339s8p’}}
2022-01-28 18:32:38 DEBUG FORWARD PAGE 8
2022-01-28 18:32:39 DEBUG https://api.twitter.com:443 “GET /2/tweets/1468347983838134274/retweeted_by?max_results=25&pagination_token=7140dibdnow9c7btw480iy7ziduqm5c5ytdlnxboa3msx HTTP/1.1” 200 1126
2022-01-28 18:32:39 DEBUG {‘data’: [‘1135536966’, ‘1301306322’, ‘4859475975’, ‘28374587’, ‘929399424’, ‘1623634472’, ‘947601462246326272’, ‘954786982043045888’, ‘1420130200747192322’, ‘2921153492’, ‘1167284611044347904’, ‘1001628052424572928’, ‘966829738714173440’, ‘17962441’, ‘162566080’, ‘873780536’, ‘756551091182002176’, ‘2811954737’, ‘1601393774’, ‘3245486234’, ‘1070870213216874496’, ‘66489136’, ‘1242222318’, ‘2557179222’, ‘601521587’], ‘meta’: {‘result_count’: 25, ‘next_token’: ‘7140dibdnow9c7btw480iy7zidmzxvb2z1qe6h9pcqgqd’, ‘previous_token’: ‘77qpymm88g5h9vqkluxbzdqc9dzqgjmiue4hwifvnvjg7’}}
2022-01-28 18:32:39 DEBUG FORWARD PAGE 9
2022-01-28 18:32:39 DEBUG https://api.twitter.com:443 “GET /2/tweets/1468347983838134274/retweeted_by?max_results=25&pagination_token=7140dibdnow9c7btw480iy7zidmzxvb2z1qe6h9pcqgqd HTTP/1.1” 200 1058
2022-01-28 18:32:39 DEBUG {‘data’: [‘909216829’, ‘1065135901’, ‘232455203’, ‘1302706150694096897’, ‘1453199468132773891’, ‘1431636953263398914’, ‘547294860’, ‘1005541’, ‘714282261869297664’, ‘440079185’, ‘1874836183’, ‘46346652’, ‘473652715’, ‘213169226’, ‘1400062189403136005’, ‘63071573’, ‘3015246739’, ‘26702284’, ‘775884921705340928’, ‘1082015423212765184’, ‘278996803’, ‘782115568065228801’, ‘120841704’, ‘2865194824’, ‘21663770’], ‘meta’: {‘result_count’: 25, ‘next_token’: ‘7140dibdnow9c7btw480iy7zidfgt1hzh8ef5aculrghg’, ‘previous_token’: ‘77qpymm88g5h9vqkluxbzdqc9drzrt9x13d7hs5kwb77d’}}
2022-01-28 18:32:39 DEBUG BACKWARD PAGE 8
2022-01-28 18:32:39 DEBUG https://api.twitter.com:443 “GET /2/tweets/1468347983838134274/retweeted_by?max_results=25&pagination_token=77qpymm88g5h9vqkluxbzdqc9drzrt9x13d7hs5kwb77d HTTP/1.1” 200 901
2022-01-28 18:32:39 DEBUG {‘data’: [‘765941712351027200’, ‘14228625’, ‘715598587816620032’, ‘19612717’, ‘49436627’, ‘2543762662’, ‘44961598’, ‘1253694008611573760’, ‘20698952’, ‘489989624’, ‘15218327’, ‘372081433’, ‘14326197’, ‘1106227004024725505’, ‘198629740’, ‘23600888’, ‘872144505172094976’, ‘17727527’, ‘18749899’, ‘1343610488081379328’, ‘384198613’, ‘14298593’, ‘1408533062’, ‘732041910877978624’, ‘571543394’], ‘meta’: {‘result_count’: 25, ‘next_token’: ‘7140dibdnow9c7btw480j0b806g77irax088znwqvd1rl’, ‘previous_token’: ‘77qpymm88g5h9vqkluxbzftv7fwv8ofkr8mkchpkokc3d’}}
2022-01-28 18:32:39 DEBUG BACKWARD PAGE 7
2022-01-28 18:32:39 DEBUG https://api.twitter.com:443 “GET /2/tweets/1468347983838134274/retweeted_by?max_results=25&pagination_token=77qpymm88g5h9vqkluxbzftv7fwv8ofkr8mkchpkokc3d HTTP/1.1” 200 691
2022-01-28 18:32:39 DEBUG {‘data’: [‘1105855292443713537’, ‘245634496’, ‘24807828’, ‘3130347809’, ‘10680442’, ‘981519459306307584’, ‘1233080592’, ‘8657052’, ‘455900925’, ‘364499002’, ‘7153662’, ‘2645987132’, ‘234760327’, ‘203219181’, ‘755352’, ‘539458560’, ‘3769484542’, ‘426039231’], ‘meta’: {‘result_count’: 18, ‘next_token’: ‘7140dibdnow9c7btw480j0bigfrxj58be6iixlzwjty60’}}

2 Likes
#3

Thank you @notpickard for taking a look at this, and for confirming the reality of this issue.

Fortunately, this issue is not blocking for me, but I was worried it could be problematic for other developers at a later point. The main benefit using a previous_token (imo) is to make sure we can easily come back later to retrieve new entries without wasting requests/resources/rate-quota. But another way of doing it could be to stop making requests as soon as we get a user id we already got earlier.

Regardless of the cases & workarounds, I think API reliability & consistency are important aspects we should all care about, so I hope it gets eventually resolved by our overly dedicated TwitterDev saviors. :smiley:

4 Likes
#4

Thanks guys for this

#5

Hi,
I noticed the same behavior.
previous_token seems to be not consistent.
I tried getting likes on some tweets and I obtained things like the following example:

1 - next_token: 7140dibdnow9c7btw480xv98837c6qw7tm3adg0wfr31t
2 - next_token: 7140dibdnow9c7btw480xv9882zkeo80k9ososphvr2gg, previous_token: 77qpymm88g5h9vqkluxcearkz3cc0ub615d2a6sb5nkfl
3 - next_token: 7140dibdnow9c7btw480xv9882rvw62gahx2hxsbaw4l1, previous_token: 77qpymm88g5h9vqkluxcearkz34k92fax11dq5xkdyywo
4 - next_token: 7140dibdnow9c7btw480xv9882k8gjsb52tn03cegge2h, previous_token: 77qpymm88g5h9vqkluxcearkz2wvqpni0liked45lppgy
5 - next_token: 7140dibdnow9c7btw480xv9882cpasycvzgeynx02h1yv, previous_token: 77qpymm88g5h9vqkluxcearkz2p8b3de9sdnvdu0g49io
6 - next_token: 7140dibdnow9c7btw480xv9882ciueb4jjmw2655b0e2f, previous_token: 77qpymm88g5h9vqkluxcearkz2hp5clpzi3x3w6b23k1f
7 - next_token: 7140dibdnow9c7btw480xv988251vho99nkb8s89t80mw, previous_token: 77qpymm88g5h9vqkluxcearkz2hioxwyllwmcx1uctp9t
8 - next_token: 7140dibdnow9c7btw480xv98824zoygfhdo0ff0elr22e, previous_token: 77qpymm88g5h9vqkluxcearkz2a1q1auokz5zxme944dt
9 - next_token: 7140dibdnow9c7btw480xv98824whih4n0ycmoytktdtg, previous_token: 77qpymm88g5h9vqkluxcearkz29zji30om2hntmh50rhx
10 - next_token: 7140dibdnow9c7btw480xv98824vd9cd5uczt3uljwkzb, previous_token: 77qpymm88g5h9vqkluxcearkz29wc2613x6fuivzjhelx
11 - next_token: 7140dibdnow9c7btw480xv98824t8umar99jwrlaejjna, previous_token: 77qpymm88g5h9vqkluxcearkz29v7t0i9s58xpxwh9e7m
12 - next_token: 7140dibdnow9c7btw480xv98824s5ytx2lwmpjbblkeux, previous_token: 77qpymm88g5h9vqkluxcearkz29t3eb7jpk7zw739zdbm
13 - next_token: 7140dibdnow9c7btw480xv98824s4qxmzuqznjddwqyb7, previous_token: 77qpymm88g5h9vqkluxcearkz29s0iitv7ob2kuxmab75
14 - next_token: 7140dibdnow9c7btw480xv9881xdcoingt4t7xjfjmskz, previous_token: 77qpymm88g5h9vqkluxcearkz29rzamjgpcf31xj3f7eb
15 - next_token: 7140dibdnow9c7btw480xv9881xdcix5yo7jb0snox7hz, previous_token: 77qpymm88g5h9vqkluxcearkz22d787k9ecuxku37uwjb
16 - next_token: 7140dibdnow9c7btw480xv9881xdcdilm3nlytvgyitdk, previous_token: 77qpymm88g5h9vqkluxcearkz22d72m2yzjdgheoqpstt
17 - next_token: 7140dibdnow9c7btw480xv9881xdcdgace6vg5y3xmofn, previous_token: 77qpymm88g5h9vqkluxcearkz22d6x7imfjm9ljja0g1c
18 - next_token: 7140dibdnow9c7btw480xv9881xdc85kqndq9laxkq628, previous_token: 77qpymm88g5h9vqkluxcearkz22d6x578xqokxtuojeia
19 - next_token: 7140dibdnow9c7btw480xv9881xdc82i7vmgw6npmg20o, previous_token: 77qpymm88g5h9vqkluxcearkz22d6ruhqy5uzh78ypbn8
20 - previous_token: 7140dibdnow9c7btw480xv9881xdc82i7vmgw6npmg20o

Except the last iteration, previous_token is always different from next_token of the previous request.
Shouldn’t previous_token always be equal to next_token of the previous request?

Hope to get this clarified soon, it’s getting me mad :slight_smile:

1 Like
#6

I confirm that I’ve also seen the same issue occurring on likes. As for the token difference, it’s actually consistent with the tokens returned in other well-working endpoints (user timeline & mentions for example): they must be different to let the API know if we’re going onwards or backwards.
+ I managed to decode them and I can confirm that they are looking to be encoded as intended.

But that was a good question to ask anyway. And thank you for confirming the issue. :slight_smile:

1 Like