Incomplete Meta data from Search API


#1

Hi guys.
I am trying to use twitter search API, but I found that most of the time, the json object retrieved has missing meta data. Like the correct expanded URL in the following case:

Actual Tweet:

In-dev BlackMist Ransomware. Targets C:\Users\Owner. Crashes. Appends blackmist to encrypted files.
https://www.virustotal.com/#/file/a7e426dc46a92d8358ab196a0f4f907b78887538cd78d447fee47b50cdc0410c/detection

linkhttps://twitter.com/i/web/status/912826078078668800

and with the search API, the following was retrieved:

{  
  contributors:None,
  truncated:False,
  text:   RT @LawrenceAbrams:   In-dev BlackMist Ransomware. Targets C:   \\Users\\Owner. Crashes. Appends blackmist to encrypted files.\nhttps://t.co/nLmrSD\u2026,
  is_quote_status:False,
  in_reply_to_status_id:None,
  id:913131435300655104,
  favorite_count:0,
  entities:{  
     symbols:[  

     ],
     user_mentions:[  
        {  
           id:15534414,
           indices:[  
              3,
              18
           ],
           id_str:15534414,
           screen_name:LawrenceAbrams,
           name:Lawrence Abrams
        }
     ],
     hashtags:[  

     ],
     urls:[  

     ]
  },
  retweeted:False,
  coordinates:None,
  source:   <a href="https://www.199lee.com" rel="nofollow">JonaHAH_</a>,
  in_reply_to_screen_name:None,
  in_reply_to_user_id:None,
  retweet_count:6,
  id_str:913131435300655104,
  favorited:False,
  retweeted_status:{  
     contributors:None,
     truncated:True,
     text:      In-dev BlackMist Ransomware. Targets C:      \\Users\\Owner. Crashes. Appends blackmist to encrypted files.\u2026 https://t.co/nm3VC2UTWO,
     is_quote_status:False,
     in_reply_to_status_id:None,
     id:912826078078668800,
     favorite_count:9,
     entities:{  
        symbols:[  

        ],
        user_mentions:[  

        ],
        hashtags:[  

        ],
        urls:[  
           {  
              url:               https://t.co/nm3VC2UTWO,
              indices:[  
                 101,
                 124
              ],
              expanded_url:               https://twitter.com/i/web/status/912826078078668800,
              display_url:twitter.com/i/web/status/9\u2026
           }
        ]
     },
     retweeted:False,
     coordinates:None,
     source:      <a href="http://twitter.com" rel="nofollow">Twitter Web Client</a>,
     in_reply_to_screen_name:None,
     in_reply_to_user_id:None,
     retweet_count:6,
     id_str:912826078078668800,
     favorited:False,
     user:{  
        follow_request_sent:False,
        has_extended_profile:False,
        profile_use_background_image:True,
        default_profile_image:False,
        id:15534414,
        profile_background_image_url_https:         https://abs.twimg.com/images/themes/theme1/bg.png,
        verified:False,
        translator_type:none,
        profile_text_color:333333,
        profile_image_url_https:         https://pbs.twimg.com/profile_images/877577333145731072/ZvJk2zVW_normal.jpg,
        profile_sidebar_fill_color:DDEEF6,
        entities:{  
           url:{  
              urls:[  
                 {  
                    url:                     http://t.co/VpBdY8GpTS,
                    indices:[  
                       0,
                       22
                    ],
                    expanded_url:                     http://www.bleepingcomputer.com/,
                    display_url:bleepingcomputer.com
                 }
              ]
           },
           description:{  
              urls:[  

              ]
           }
        },
        followers_count:904,
        profile_sidebar_border_color:C0DEED,
        id_str:15534414,
        profile_background_color:C0DEED,
        listed_count:32,
        is_translation_enabled:False,
        utc_offset:-14400,
        statuses_count:289,
        description:Ransomware,
        Online Security,
        and Malware. Wish I had more time for gaming. Owner of @bleepincomputer.,
        friends_count:66,
        location:New York,
        profile_link_color:1DA1F2,
        profile_image_url:         http://pbs.twimg.com/profile_images/877577333145731072/ZvJk2zVW_normal.jpg,
        following:False,
        geo_enabled:False,
        profile_background_image_url:         http://abs.twimg.com/images/themes/theme1/bg.png,
        screen_name:LawrenceAbrams,
        lang:en,
        profile_background_tile:False,
        favourites_count:151,
        name:Lawrence Abrams,
        notifications:False,
        url:         http://t.co/VpBdY8GpTS,
        created_at:         Tue Jul 22 17:59:55         +0000 2008,
        contributors_enabled:False,
        time_zone:Eastern Time (US & Canada),
        protected:False,
        default_profile:True,
        is_translator:False
     },
     geo:None,
     in_reply_to_user_id_str:None,
     possibly_sensitive:False,
     lang:en,
     created_at:      Tue Sep 26 23:47:51      +0000 2017,
     in_reply_to_status_id_str:None,
     place:None,
     metadata:{  
        iso_language_code:en,
        result_type:recent
     }
  },
  user:{  
     follow_request_sent:False,
     has_extended_profile:False,
     profile_use_background_image:False,
     default_profile_image:False,
     id:758329841297059840,
     profile_background_image_url_https:      https://abs.twimg.com/images/themes/theme1/bg.png,
     verified:False,
     translator_type:none,
     profile_text_color:000000,
     profile_image_url_https:      https://pbs.twimg.com/profile_images/758332767209947136/Zewxa_mL_normal.jpg,
     profile_sidebar_fill_color:000000,
     entities:{  
        description:{  
           urls:[  

           ]
        }
     },
     followers_count:3154,
     profile_sidebar_border_color:000000,
     id_str:758329841297059840,
     profile_background_color:000000,
     listed_count:1426,
     is_translation_enabled:False,
     utc_offset:None,
     statuses_count:21731,
     description:r00ted boxes back in the day,
     now i lurk in irc.,
     friends_count:4287,
     location:errwhere,
     profile_link_color:ABB8C2,
     profile_image_url:      http://pbs.twimg.com/profile_images/758332767209947136/Zewxa_mL_normal.jpg,
     following:False,
     geo_enabled:False,
     profile_banner_url:      https://pbs.twimg.com/profile_banners/758329841297059840/1469636620,
     profile_background_image_url:      http://abs.twimg.com/images/themes/theme1/bg.png,
     screen_name:JonaHAH_,
     lang:en,
     profile_background_tile:False,
     favourites_count:2197,
     name:JonaHAH,
     notifications:False,
     url:None,
     created_at:      Wed Jul 27 15:54:58      +0000 2016,
     contributors_enabled:False,
     time_zone:None,
     protected:False,
     default_profile:False,
     is_translator:False
  },
  geo:None,
  in_reply_to_user_id_str:None,
  lang:en,
  created_at:   Wed Sep 27 20:01:13   +0000 2017,
  in_reply_to_status_id_str:None,
  place:None,
  metadata:{  
     iso_language_code:en,
     result_type:recent
  }
}

#2

Can you provide the exact search query you issued to retrieve that Tweet ID?


#3

The exact link was:
https://api.twitter.com/1.1/search/tweets.json/?lang=en&q=%23virustotal.com%20&count=100&src=typd


#4

You should use the tweet_mode=extended parameter.


#5

Thanks it worked.