Emoji characters break the accuracy of entity indice values

[sebelrs]

This has been raised in another topic in 2017 but seemingly no answer yet.

Can't work out entity indices in specific tweet with emojis

Hello. Did you ever get a reply or a solution for this problem? I’m about to post the same question.

I’m using twitteroauth for PHP.

The indices for hashtags etc become unreliable when emojis are present in a tweet. Seemingly the emoji is counted as one character, but this character does not seem to be taken in to account when calculating in and out indices for hastags, user_mentions etc.

In the example below, the indices returned in the JSON block will select ’ @bcui’ instead of ‘@bcuic’ because of the 1 character seemingly counted by the emoji at the start of the tweet, but not accounted for in the indice values

\ud83c\udf40Good luck to our UK colleges @bcuic & …ect

Here’s one more example with a screen shot of my render.
“full_text”:“Will be interesting to see how the #AFL suspends Toby Greene this week…\ud83e\udd14 #getcreative #AFLFinals”

“text”:“getcreative”,
“indices”:[
75,
87
]

Should I look to strip out emojis from the fullPtext result?
Should I follow up the links on this page…

New update to the Twitter-Text library: Emoji character count

Over the years, we have made several updates to the way that people can communicate on Twitter. One of the more notable changes made last year was to increase the number of characters per Tweet from 140 to 280 characters. Today, we continue to expand people’s ability to express themselves by announcing a change to the way that we count emojis. Due to the differences in the way written text and emojis are encoded, many emojis (including emojis where you can apply gender and skin tone) have count…

[andypiper]

The algorithms documented and implemented in the twitter-text library are the correct way to handle counting emoji characters. We don’t currently ship a PHP implementation of this algorithm, but it should be possible to derive one from the other code samples.