Truncated statuses using Standard Search API with npm twitter package

nodejs
javascript
api

#1

I’d like to know if anyone is able to retrieve full-text, non-truncated tweets using the Standard API.

Using npm twitter package to search tweets, I’m getting truncated tweets even when I set truncated: false. Below is a toy example of the data object of an individual tweet. As you can see, the text is truncated at the end and the missing text is replaced by this elipses ...

I’ve read the docs and googled extensively, and the commonly-recommended solutions don’t work for me. There are no tweet_mode: 'extended' or extended_tweet or retweeted_status or full_text anywhere in this data object, which is probably why nothing is return when I include these in the API call parameters.

Is this a limitation of the library, am I doing something wrong, or are full tweets only available in paid versions of the API? Because in this article the sample JSON from the Enterprise API (paid) does contain the data nodes required to retrieve full-text tweets.

My code is included below this data object. I’d like to know if anyone has a better solution, using any tool. Maybe I should try cURL. I haven’t figured out the full script yet, so I’m wondering if anyone has already done this before and can share.

{ statuses:
   [ { created_at: 'Mon Jan 15 06:10:19 +0000 2018',
       id: 952784997106139100,
       id_str: '952784997106139137',
       text: 'RT @amjoyshow: .@RepMaxineWaters on GOP colleagues: They cannot credibly come before the American public and defend #DonaldTrump. They’re a…',
       truncated: false,
       entities: [Object],
       metadata: [Object],
       source: '<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>',
       in_reply_to_status_id: null,
       in_reply_to_status_id_str: null,
       in_reply_to_user_id: null,
       in_reply_to_user_id_str: null,
       in_reply_to_screen_name: null,
       user: [Object],
       geo: null,
       coordinates: null,
       place: null,
       contributors: null,
       retweeted_status: [Object],
       is_quote_status: false,
       retweet_count: 1195,
       favorite_count: 0,
       favorited: false,
       retweeted: false,
       lang: 'en' } ],
  search_metadata:
   { completed_in: 0.019,
     max_id: 952784997106139100,
     max_id_str: '952784997106139137',
     next_results: '?max_id=952784997106139136&q=%2523donaldtrump&geocode=40.7127837%2C-74.0059413%2C10mi&lang=en&count=1&include_entities=1&result_type=recent',
     query: '%2523donaldtrump',
     refresh_url: '?since_id=952784997106139137&q=%2523donaldtrump&geocode=40.7127837%2C-74.0059413%2C10mi&lang=en&result_type=recent&include_entities=1',
     count: 1,
     since_id: 0,
     since_id_str: '0' } }

If you want to reproduce the results use this, though you’ll need to make a separate config.js file to hold your authentication keys:

const Twitter = require('twitter');
const config = require('./config.js'); // Separate file that contains the 4 authentication keys
const fs = require('fs');

var client = new Twitter(config); 
var params = {
	q: '%23donaldtrump', // Hashtag
	geocode: '40.7127837,-74.0059413,10mi', // New York City
	count: 1,
	result_type: 'recent',
	lang: 'en',
	retweeted: false,
	truncated: false,
	retweeted_status:
	{
		truncated: false
	}
}

client.get('search/tweets', params, (err, data) => {
	if(err) throw err;
	console.log(data);
});

#2

You should try including tweet_mode in your params values. The other values you’ve included in there are JSON response values, not valid parameters for calling the API, so we can get rid of those.

var params = {
	q: '%23donaldtrump', // Hashtag
	geocode: '40.7127837,-74.0059413,10mi', // New York City
	count: 1,
	result_type: 'recent',
	lang: 'en',
	tweet_mode: 'extended'
}

#3

I did try that. this did not work because the data object itself did not contain the parameter tweet_mode, as you can see in the complete data object returned, in the first code block I shared.


#4

That would make sense, because tweet_mode is an API call parameter, not a data field.

I’ve just tried my version of the code here, and the full_text field does appear in the returned Tweet(s), indicating that the tweet_mode parameter is being passed through to the API. I get back non-truncated Tweets with the text in the full_text field.


#5

The returned text parameter is now labeled full_text, but the text returned is still truncated. Is there a different parameter for retrieving all 280 characters of a tweet, not just 140 characters?

{ created_at: 'Mon Jan 15 15:08:41 +0000 2018',
       id: 952920478041215000,
       id_str: '952920478041214977',
       full_text: 'RT @DavidRomeiPHD: #DonaldTrump knows that #TomCotton of #Arkansas would set naked on a block of ice if he was asked to do so by his Lord #…',
       truncated: false,
       display_text_range: [Object],
       entities: [Object],
       metadata: [Object],
       source: '<a href="https://mobile.twitter.com" rel="nofollow">Twitter Lite</a>',
       in_reply_to_status_id: null,
       in_reply_to_status_id_str: null,
       in_reply_to_user_id: null,
       in_reply_to_user_id_str: null,
       in_reply_to_screen_name: null,
       user: [Object],
       geo: null,
       coordinates: null,
       place: null,
       contributors: null,
       retweeted_status: [Object],
       is_quote_status: false,
       retweet_count: 1,
       favorite_count: 0,
       favorited: false,
       retweeted: false,
       lang: 'en' },
     { created_at: 'Mon Jan 15 15:08:29 +0000 2018',
       id: 952920429794091000,
       id_str: '952920429794091008',
       full_text: 'RT @amjoyshow: .@DRCOREYHEBERT gives his expert view on #DonaldTrump’s health: \n\nAny clinician with a common sense attitude at all could de…',
       truncated: false,
       display_text_range: [Object],
       entities: [Object],
       metadata: [Object],
       source: '<a href="http://twitter.com/download/android" rel="nofollow">Twitter for Android</a>',
       in_reply_to_status_id: null,
       in_reply_to_status_id_str: null,
       in_reply_to_user_id: null,
       in_reply_to_user_id_str: null,
       in_reply_to_screen_name: null,
       user: [Object],
       geo: null,
       coordinates: null,
       place: null,
       contributors: null,
       retweeted_status: [Object],
       is_quote_status: false,
       retweet_count: 192,
       favorite_count: 0,
       favorited: false,
       retweeted: false,
       lang: 'en' },

#6

This specific Tweet is a retweet, so the actual text is inside the retweeted_status object. I just pulled the same Tweet using twurl and the text in the top level Tweet is truncated with “RT” prepended, but the retweeted_status->full_text field is complete.


#7

Thanks for your responses, they’ve been very helpful. I just want to clarify something; let me know if I’m right or wrong. This is how I see it:

  1. Original tweets always have complete .full_text properties.
  2. Retweets always have truncated .full_text properties but complete .retweet_status.full_text properties.

Is that correct? The following is the console log of a retweet object. Its .full_text property is truncated with elipsis ... but it has a complete .retweet_status.full_text property (the property is not displayed in the code here but I checked and it was complete):

{ created_at: 'Mon Jan 15 23:32:12 +0000 2018',
       id: 953047194155798500,
       id_str: '953047194155798528',
       full_text: 'RT @amjoyshow: .@REVJAMESFORBES tells #AMJoy how he had a "broken heart attack" upon hearing the #DonaldTrump s***hole reports, but hopes w…',
       truncated: false,
       display_text_range: [Object],
       entities: [Object],
       metadata: [Object],
       source: '<a href="http://twitter.com/download/iphone" rel="nofollow">Twitter for iPhone</a>',
       in_reply_to_status_id: null,
       in_reply_to_status_id_str: null,
       in_reply_to_user_id: null,
       in_reply_to_user_id_str: null,
       in_reply_to_screen_name: null,
       user: [Object],
       geo: null,
       coordinates: null,
       place: null,
       contributors: null,
       retweeted_status: [Object],
       is_quote_status: false,
       retweet_count: 537,
       favorite_count: 0,
       favorited: false,
       retweeted: false,
       lang: 'en' }

#8

Yes, that’s correct.