Pulling data with api.search and some "full text" is short cut

[personalizedCo]

Hi,
I am using api.search to pull data related to a certain hashtag. Then I am writing the data csvWriter.writerow([tweet.created_at, **tweet.text.encode('utf-8')**])

However, the text is short cut for certain text. I know it is because of the certain type of characters for example if there is “:” then it will encode it. Instead of “5:20” you get “5\xe2\x80\xa6” and the rest of the tweet disappears.

If I do not use encode only say **tweet.text** then it gets even worse with awkward characters like üëèüëè

Thank you!

[jessicagarson]

Thanks for reaching out here. Would you be able to send over a longer code sample? Also are you using the search tweets Python wrapper?

[personalizedCo]

Hi @jessicagarson I am using Tweepy. Here it is the longer code:

for tweet in tweepy.Cursor(api.search,q= "#iteachmath" or "#MTBoS", count=100,
                           lang="en",
                           since="2017-01-01").items():
    csvWriter.writerow([tweet.created_at, tweet.text.encode('utf-8'), tweet.user, tweet.retweet_count, tweet.favorite_count, tweet.entities, tweet.coordinates])

Another issue I am running into is related to Twitter Archive. Initially I asked it here and they redirected me to the Twitter Support and Twitter Support was not helpful at all. When I download my archive and open the csv almost all the tweets are short cut with “Ķ” this sign. Attached you will see the example of it:

Thank you!

[andypiper]

Unfortunately the Twitter Archive feature is not a part of the developer platform so we cannot help with this specific issue here, but I will pass this issue along to the team.

[personalizedCo]

I found out why I am facing the issue and why Twitter archive too has the same problem. Both you guys and I are using the wrong key. “text” for some reason will always cut short but if you use “retweeted_status” and from there pull the “text”, you can get the full text with no problem. Thank you!

[LeBraat]