When filtering for `native_video`, many tweets do not contain `extended_entities`

[ffshr]

Hi!

I’m using the standard twitter search API via the python Twython library to search for native videos. I noticed that frequently, the tweets in the results do not contain extended_entities, sometimes they do at a later time, but not always. I’m using the tweet_mode=extended parameter, but this doesn’t rectify the issue fully.

As example, searching for hurricane filter:native_video -filter:retweet I’m getting 214 tweets with extended_entities but 786 without.

Is this expected behavior? Is there a way to improve the result completeness?

Thanks in advance,
@ffshr

[hmz]

Hey @ffshr.

I’d also want to improve the completeness of my results if it was giving me data I didn’t need. Could you please provide an example tweet which DOESN’T have the extended_entities so I can find out why this is happening:

I’m getting 214 tweets with extended_entities but 786 without.

Hamza

[ffshr]

Hi Hamza!

It turns out it was my fault: I missed the plural s when filtering retweets, hence a lot of retweets were in the result set, and there the extended_entities is nested in the retweeted_status.

adding the s now results in 986 out of 1000 with extended_entities. One of the few without the extended entities is, e.g., https://twitter.com/MyRadarWX/status/1053057876825202690

best,
@ffshr

[hmz]

It looks like that particular tweet is truncated. Could you try adding tweet_mode=extended to your query to get back the full tweet body?

Let me know if that returns the results you were wanting.

Hamza

[ffshr]

Hi!

I am already using tweet_mode=extended. For my use case the current result is OK. I don’t need it necessarily in 100% of the tweets, 99% is good enough.

Best,
@ffshr

[system]

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.