Missing images randomly


#1

Hello,

I’m using the streaming API to get notified of tweets for a specific account in real time.
However, for some reason, the images are not sent for some tweets, randomly.
Can I do something about it ?

Thank you.


#2

Do you have specific Tweet IDs that appear not to include images when they should have done so? It is not very clear from the screenshot you’ve shared here what is going on. Can you explain exactly what your code is doing and why you believe images are “not sent”?


#3

Sure,

The tweet missing the image (the first one in my screenshot) is https://twitter.com/DayZ/status/1082894742512431104
My code is looking for the extended_entities key, then the media key, and if both are present it iterates over the media array. Then, if the media checked is a photo, it removes the media’s url from the tweet text and adds an image to a Discord richembed (you can see two differents RichEmbeds in my screenshot, with the blue left border), then send the said richembed in a Discord channel.
I also tried with thetweet_mode=extended parameter, did not fixed it.

I just added extra logging as I can’t reproduce if I follow my own account and tweet images, so I can provide you more infos


#4

I’ve just retrieved that Tweet ID using tweet_mode=extended and the media entities show up as expected. Note that, for Tweets from the Streaming API, this parameter does not work; instead, the API serves up both versions of the Tweet in the same object. You should be able to find the extended Tweet content inside the main Tweet.


#5

I just found out why I’m missing some images.

In your documentation (https://developer.twitter.com/en/docs/tweets/data-dictionary/overview/extended-entities-object.html), you say that “If a Tweet contains native media (shared with the Tweet user-interface as opposed via a link to elsewhere), there will also be a extended_entities section.”. But for some reason, sometimes the extended_entities is in the root object, sometimes it is inside the extended_tweet object, and that’s where I’m missing images.
For example, today I missed this tweet image https://twitter.com/DayZ/status/1084797451842785281 because the object I got from the streaming API was

{ created_at: 'Mon Jan 14 13:00:41 +0000 2019',
  id: 1084797451842785300,
  id_str: '1084797451842785281',
  text: 'XBOX PLAYERS:\n\nWe have a new update for you, including some important fixes for weapon reloading and unjamming. The… https://t.co/BLugcQLz7W',
  display_text_range: [ 0, 140 ],
  source: '<a href="https://about.twitter.com/products/tweetdeck" rel="nofollow">TweetDeck</a>',
  truncated: true,
  in_reply_to_status_id: null,
  in_reply_to_status_id_str: null,
  in_reply_to_user_id: null,
  in_reply_to_user_id_str: null,
  in_reply_to_screen_name: null,
  user: 
   { id: 550528636,
     id_str: '550528636',
     name: 'DayZ - PC 1.0 out now!',
     screen_name: 'DayZ',
     location: 'Chernogorsk, Chernarus',
     url: 'http://www.dayz.com',
     description: 'Official Twitter account for #DayZ by @bohemiainteract',
     translator_type: 'none',
     protected: false,
     verified: true,
     followers_count: 229051,
     friends_count: 216,
     listed_count: 837,
     favourites_count: 6205,
     statuses_count: 16683,
     created_at: 'Tue Apr 10 23:26:54 +0000 2012',
     utc_offset: null,
     time_zone: null,
     geo_enabled: true,
     lang: 'en',
     contributors_enabled: false,
     is_translator: false,
     profile_background_color: '000000',
     profile_background_image_url: 'http://abs.twimg.com/images/themes/theme1/bg.png',
     profile_background_image_url_https: 'https://abs.twimg.com/images/themes/theme1/bg.png',
     profile_background_tile: false,
     profile_link_color: 'D81001',
     profile_sidebar_border_color: 'FFFFFF',
     profile_sidebar_fill_color: 'DDEEF6',
     profile_text_color: '333333',
     profile_use_background_image: true,
     profile_image_url: 'http://pbs.twimg.com/profile_images/1070731951353548800/f_glBn7m_normal.jpg',
     profile_image_url_https: 'https://pbs.twimg.com/profile_images/1070731951353548800/f_glBn7m_normal.jpg',
     profile_banner_url: 'https://pbs.twimg.com/profile_banners/550528636/1544728003',
     default_profile: false,
     default_profile_image: false,
     following: null,
     follow_request_sent: null,
     notifications: null },
  geo: null,
  coordinates: null,
  place: null,
  contributors: null,
  is_quote_status: false,
  extended_tweet: 
   { full_text: 'XBOX PLAYERS:\n\nWe have a new update for you, including some important fixes for weapon reloading and unjamming. The servers are currently restarting with the new version, there shouldn\'t be any wipes. You can read the patchnotes here: https://t.co/neiFbdg8cN\n\n📷@Maurotto (reddit) https://t.co/uP6xKKitOl',
     display_text_range: [ 0, 279 ],
     entities: 
      { hashtags: [],
        urls: [Array],
        user_mentions: [Array],
        symbols: [],
        media: [Array] },
     extended_entities: { media: [Array] } },
  quote_count: 0,
  reply_count: 0,
  retweet_count: 0,
  favorite_count: 0,
  entities: 
   { hashtags: [],
     urls: [ [Object] ],
     user_mentions: [],
     symbols: [] },
  favorited: false,
  retweeted: false,
  possibly_sensitive: false,
  filter_level: 'low',
  lang: 'en',
  timestamp_ms: '1547470841738' }

and as you can see the extended_entities is not at the root level, but nested inside extended_tweet.

Is that the correct behavior ? How should I know if I should use the root one, or the one in extended_tweet ? Thank you !


#6

This is correct. If the root Tweet contains truncated: true then you will want to look into the extended_tweet object for the media.


#7

Okay, thank’s for clarifying and helping me out !


closed #8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.