0 status after a 'next_result' request

[maximeburri]

Hi guys,

I try to get a lot of tweets with the query “#tpmp1 OR #tpmp2 OR #tpmp3” and I use abraham’s twitteroauth in PHP . For the first request I get 100 stasuses and i get back the ‘max_id’ from the ‘next_result’ url for the second request, and I directly send that. But I receive 0 statuses with count = 100.

There is the result of the second request.

{
   "statuses":[],
   "search_metadata": {
      "completed_in":0.004,
      "max_id":507214059847311359,
      "max_id_str":"507214059847311359",
      "query":"%2523tpmp1%2BOR%2B%2523tpmp2%2BOR%2B%2523tpmp3%2Bexclude%3Aretweets",
      "refresh_url":"?since_id=507214059847311359&q=%2523tpmp1%2BOR%2B%2523tpmp2%2BOR%2B%2523tpmp3%2Bexclude%3Aretweets&result_type=recent",
      "count":100,
      "since_id":0,
      "since_id_str":"0"
   }
}

But I noticed that if I wait for exactly 17 seconds before the second request, I receive well the 100 expected statuses (not less, 16.5 doesn’t work). Is it a way of avoiding the abuses ?

Thanks

[edsu]

I am noticing this too. I also left previous comments indicating that I can replicate it with this Python script but those comments appear to have been deleted? Maybe the comments are still there and I just can’t see them for some reason. I thought I would comment again since this continues to be a pretty nasty problem with the search api, and hopefully it will get the attention of the Twitter API development team.

[maximeburri]

I can’t see the comments too so I repeat and complete.

I noticed an even stranger thing: if the second request is directly executed and the parameter ‘count’ is strictly lower than a certain value, the second request works as expected… But this value changes… For me, on September 8th for the query ‘dconstruct’, this value was 58 but for @edsu it was 83. On 8th for me 61 and now 95. Soon it will arrive at 100 and the second request will have to work as expected…

That makes me think that the problem is caused by the parameter ‘count’ for some hashtags/queries which are not very popular or which were not ‘used’ a lot for some time, like ‘dconstruct’ or ‘#tpmp1 OR #tpmp2 OR #tpmp3’.

[maximeburri]

There’s still the issue with the query ‘dconstruct’. I found others queries with which there is the same issue, and I looked for the value for the count parameter with which the request works as expected :

• dconstruct
  • is fixed with count to 91
• #TEDxBern
  • is fixed with count to 76 or 77 (it depends)
• Switzerland AND chocolate (only Switzerland works)
  • is fixed with count to 23 or 24 (it depends)

If I waits for some time between request (~17-20 seconds) it’s works as expected.
There is an exemple of results with #TEDxBern query.

---

#TEDxBern with count setted to 100
Issue for the second request : only one status

// First request 
{
    "statuses": [
		/* 100 stasuses
		   from id 510449020201107456 
		   ...
		   to id 510195833497976832*/
	],
    "search_metadata": {
        "completed_in": 0.069,
        "max_id": 510449020201107456,
        "max_id_str": "510449020201107456",
        "next_results": "?max_id=510195833497976831&q=#TEDxBern&count=100&include_entities=1",
        "query": "#TEDxBern",
        "refresh_url": "?since_id=510449020201107456&q=#TEDxBern&include_entities=1",
        "count": 100,
        "since_id": 0,
        "since_id_str": "0"
    }
}

// Second request
{
    "statuses": [
		/* only 1 stasus
		   id 510195833497976832 */],
    "search_metadata": {
        "completed_in": 0.009,
        "max_id": 510195833497976832,
        "max_id_str": "510195833497976832",
        "query": "#TEDxBern",
        "refresh_url": "?since_id=510195833497976832&q=#TEDxBern&include_entities=1",
        "count": 100,
        "since_id": 0,
        "since_id_str": "0"
    }
}

#TEDxBern with count setted to 77
Second request works as expected

// First request
{
    "statuses": [ 
		/* 77 stasuses
		   from id 510449020201107456 
		   ...
		   to id 510195873008328704*/
	],
    "search_metadata": {
        "completed_in": 0.154,
        "max_id": 510449020201107456,
        "max_id_str": "510449020201107456",
        "next_results": "?max_id=510195873008328703&q=#TEDxBern&count=77&include_entities=1",
        "query": "#TEDxBern",
        "refresh_url": "?since_id=510449020201107456&q=#TEDxBern&include_entities=1",
        "count": 77,
        "since_id": 0,
        "since_id_str": "0"
    }
}

// Second request
{
    "statuses": [
		/* 77 stasuses
		   from id 510195873008328704 
		   ...
		   to id 510101731829903360*/
		   ],
    "search_metadata": {
        "completed_in": 0.05,
        "max_id": 510195873008328704,
        "max_id_str": "510195873008328704",
        "next_results": "?max_id=510101731829903359&q=#TEDxBern&count=77&include_entities=1",
        "query": "#TEDxBern",
        "refresh_url": "?since_id=510195873008328704&q=#TEDxBern&include_entities=1",
        "count": 77,
        "since_id": 0,
        "since_id_str": "0"
    }
}

---

Do you have the same results as @edsu and me? Thank you in advance and sorry for my english.

Maxime Burri

[edsu]

I’m still seeing the same problem with the second set of results. It’s pretty trivial to replicate the problem, but if you need an example checkout out this short python script.

[andypiper]

Hey everyone - thanks for the report on this. We’ve reproduced internally and have a ticket with the Search team to look into the issue. Apologies for the confusion, and good sleuthing on identifying the issue!

[andypiper]

This issue has now been resolved - we appreciate your patience!

[andypiper]