False positives on Duplicate tweets


It looks like the duplicate rules have changed for the tweet api. We run an app that performs tweets for users with a single click. Over the last couple of days we’re seeing tweets being returned as “duplicates” from the API. For example:
"#RT I’ve just entered to #win a #fairykit from @MyOwnFairyLtd with @MyFamilyTies1 #competition https://wn.nr/Hm3yY"
was considered to be a duplicate of
"Win a Fairy Kit from @MyOwnFairyLtd @MyFamilyTies1 #win #competition #giveaway #Fairy https://wn.nr/Hm3yY"

Even more odd is this:
We tried to tweet this:
"FIVE WAYS TO ANNOY PEOPLE! It’s easy to confuse and annoy people online - You won’t BELIEVE number 4! http://dannyuk.com/2015/11/12/how-to-annoy-idiots-online-you-wont-believe-number-4/"
and this got returned as a duplicate:
"I’ve just entered to win a @Bratz_UK #FierceFitness Yasmin Doll thanks to @mummy2monkeys1 #bratz #doll https://wn.nr/K95fU "

Looking at the times, I’ve had an instance where a tweet has been returned as a duplicate for five completely different tweets over the space of a few minutes. It looks like if an app sends six different tweets for a user in three minutes are they all considered duplicates.

Has this changed recently? It seems very broken. Is there any way this could be fixed so that very different tweets aren’t considered to be the same?

Invalid detection of duplicated tweets

Looks like a bug in the ruby gem, not the API itself: https://github.com/sferik/twitter/issues/743


Seems someone else has similar issues: Has the Duplicate Tweet rules changed? See 10x increase in rejections