Retrieve full tweet when truncated (non-retweet)

[MrTurnerj]

So from what I have read (and what would seem obvious to others), Twitter can and does truncate tweets that meet certain conditions - this is completely fine and understandable. What I am having a bit of trouble with is actually retrieving the full tweet itself when it is truncated.

Sound confusing? Here is an example, look up this tweet (id: 782845350918971393). It isn’t a retweet itself and the Web UI displays the full message but when accessing it from the API, I get this back:

A #Gameofthrones fan? Our #earlybird Dublin starter will get you touring the GOT location in 2017 #traveldealls… https://t.co/v8AZKrEokz

The API is saying this tweet is truncated, the text itself is truncated and the URL points back to the web UI. Beyond scraping the tweet from the Web UI, does the API not return the un-truncated version?

I’m specifically wanting to get the real message, I want this from the API (note the different URL in the tweet):

A #Gameofthrones fan? Our #earlybird Dublin starter will get you touring the GOT location in 2017 #traveldealls https://t.co/Jtur98U8zv

I’ve been experimenting with Apigee to see if it was the endpoint I was using has this problem (specifically, I was using the search endpoint) however I have been able to test that it also affects the main tweet retrieval endpoint (/statuses/show)

Everything I have searched about truncation refers to a retweet being truncated but this isn’t a retweet!

[IgorBrigadir]

Try it with &tweet_mode=extended parameter in the call - should get the full tweet that way

[MrTurnerj]

Wow, I’ve been staring at the documentation for hours and didn’t notice that parameter! I can confirm it does exactly what I need!

Thank you very much!

For anyone else that reads this post, look at the upcoming changes to tweets page in the documentation and search for “tweet_mode”.

[LindseyBass]

Hello guys,

I am trying to download tweets using tweepy with tweet_mode=extended and my query looks like this:

tweepy.Cursor(api.search,q='to%3ANASA&tweet_mode=extended').items(limit)

, but I keep getting zero results. As soon as I remove the &tweet_mode=extended part from the query, I get plenty of results:

tweepy.Cursor(api.search,q='to%3ANASA').items(limit)

Is it still the right way to form a query? Any idea why am I not getting anything with extended mode? I have been searching all over the web to find any hints or updates, but this is the only relevant discussion I was able to find… Hope you can help!

Thanks so much,
Lindsey

[IgorBrigadir]

When using tweepy, the tweet_mode=extended part of the call is set like this:

tweepy.Cursor(api.search,q='to%3ANASA', tweet_mode='extended').items(limit)

That should work!

[LindseyBass]

Thank you Igor, yes that worked!

[VijayShGupta]

Is , it possible in R, Using twitteR package…

[andypiper]

Yes, it should be, if you add the tweet_mode=extended parameter.

[VijayShGupta]

Hello Andypiper,

Thanks for the reply.

It’s not working, I tried the same.

tweets<-searchTwitter(“xxxxxxx”,n=25,tweet_mode=“extended” );

Error:

Error in tw_from_response(out, …) :

unused argument (tweet_mode = “extended”)

Best,

Vijay Gupta

[andypiper]

In that case I suggest you’d need to raise this with the authors of the twitteR package to find out whether this parameter is supported or not.

[AlexF]

I’m trying the same thing. I have copied and pasted:

tweepy.Cursor(api.search,q=‘to%3ANASA’, tweet_mode=‘extended’).items(limit)

It makes sense and matches with the twitter API, but the problem is that, although the tweets returned have the field ‘full_text’ instead of ‘text’, the full text is also truncated with the … at the end.

I have been researching about this problem for hours and I don’t know how to solve it. Need your help

Thankyou

[andypiper]

Could there be anything that Tweepy itself is doing to truncate the data?

[AlexF]

I don’t think so, because I tried it ‘by hand’ with outh and urlib and the result is the same.

It works only with the streaming.
According to: https://dev.twitter.com/overview/api/upcoming-changes-to-tweets the streaming works with the method in the second row of that table, while I’m trying to use without success the method of the third row for the search of tweets.

[andypiper]

Sorry I’m confused. Can you provide a Tweet ID so I can confirm using twurl?

[AlexF]

Of course. For instance, try this: 910421857354043392

[andypiper]

Thanks.

It’s a retweet, so the actual full_text field is inside the retweeted_status object further down inside the Tweet object.

[AlexF]

910421857354043392

Thankyou!

How would you get it with api.search? I mean, the retweeted_status

[AlexF]

Sorry, I replayed myself hahaha

[AlexF]

Ok, definitely it only happend with retweets. I didn’t notice until now. Thanks a lot. I leave a litle extract of code just in case anyone has the same problem as I’ve had

for tweet_info in tweepy.Cursor(api.search, q=query, lang = ‘en’, tweet_mode=‘extended’).items(100):
if ‘retweeted_status’ in dir(tweet_info):
tweet=tweet_info.retweeted_status.full_text
else:
tweet=tweet_info.full_text

[andypiper]