Inconsistent results via search


#1

Not sure where to look at now. I’m using Python and Tweepy to do the search and observing a weird behaviour. Sometimes (I cannot see the pattern) the search returns 1700 results, sometimes 400 results, sometimes 100. The code is exactly the same and all that changes is the time when I execute the script. It can be now 200 results, and 5 minutes later 1000 results, then after few minutes only 100.
However when I’m trying to do the search via Twitter API console the results are always consistent and just as I expect them.

I tried to see the urls that are being queried by the library. The library request receives empty response from Twitter, exactly the same url via Twitter API console receives full page of results.

Response headers look sane, I’m not being blocked or rate-limited. I also tried to change the app, reset tokens. Same result.


#2

Has anyone experienced similar results?


#3

I managed to find the issue myself. Not sure exactly why as I haven’t been able to trace it back to either my code or Tweepy, but when you set count to maximum value of 100 for the search endpoint you start getting these weird results. The same behaviour was seen for the number between 80-100.

When I have set the count to 75 all these problems disappeared.


#4

I have the same problem and facing same random behaviour of yours but i’m using Twitter4j instead.
I either get 100 results (let’s say i set the page size = 100) or all the results.
I tried with the magic number but it didn’t work for me.
Why the search call is so indeterministic? : )


#5

Have you tried reducing it even further to, say, 10 or 20? This behaviour is very frustrating


#6

Wow, i tried with 10 and it worked! now i got 506 for a user i always got max 100.
This is even more frustrating cause i’m gonna finish my calls due to the limits the API have.
In this case it took 56 calls when usually i’d have needed 6.
Thank you by the way :slight_smile:


#7

Interestingly different accounts have different values. I’ve got another account which doesn’t work for anything higher than 25.

Oh Twitter, why you doing this to us?!