Streaming API missing some timeline, replies and own tweets

lgladdy · 2013-04-26T18:17:44.000Z

If you take a look at @gmeddev’s feed I just wrote some code to send tweets to help me demonstrate this to you.

Tweets #11 and #15 didn’t appear in my userstream.

kurrik · 2013-04-26T19:00:59.000Z

Thanks, that’s helpful.

lgladdy · 2013-04-26T20:01:35.000Z

Not sure if you’ve fixed anything, but right now this issue isn’t happening. I’ve written a twitter app that does nothing but prove the existence (or not) of this bug…

It opens a userstream connection, then sends 25 tweets and makes sure those tweets appear in the feed.

So either you’ve fixed it, or this only happens at certain times, and isn’t a consistent thing. I’ll keep an eye on it, and if i notice it happening again I’ll drop you a tweet with a link to my app that shows you all the IDs.

kurrik · 2013-04-26T21:24:19.000Z

I think it may be an intermittent thing. I’ll be testing at various times with a similar approach, thanks.

gordonchiam · 2013-04-29T03:08:14.000Z

We also experienced this issue two days ago and about about a week ago. Some @replies and own tweets were missing from the user stream.

We noticed this issue after switching to v1.1 of user stream recently.

0xabad1dea · 2013-04-29T19:07:00.000Z

Also experiencing this sporadic omission of tweets while using Tweetbot and after discussing it with the author he sent me here since it seems to be the API itself…

Specific examples: https://twitter.com/jvanegue/status/328941490552311808 (private) he mentions me, I follow him, and it doesn’t show up

https://twitter.com/0xabad1dea/status/328930161854656512 my own tweet, never shows up in timeline, yet is visible on profile

Both show up as expected in official iPhone client.

I’m a heavy tweeter with a LOT of replies, may or may not be relevant

JeffMusk · 2013-04-29T23:41:55.000Z

Happened to me now for two weeks aaaaaa. And just happened right now saw the tweet roll over at top of screen but not in my mentions …please help fix it …

lgladdy · 2013-04-30T19:33:00.000Z

Can we open an issue for this now, @Kurrik?

Any kind of confirmation at least that you’re aware this is a bug and working to fix it? It’s actually a major issue in my opinion!

I’d like to get issue #999 reopened for this if possible. https://dev.twitter.com/issues/999

kurrik · 2013-05-01T01:38:03.000Z

We’re still looking into this. I don’t have an update for you, unfortunately. I’ll reopen the issue for the time being until I have more information.

dcftt · 2013-05-01T02:28:38.000Z

I ran an analysis on a subset of the Twitter users that I follow via the streaming API. Using the REST API, I pulled the timelines of these users and compared them to what I received via the streaming API. Here are my results:

First dropped tweet was encountered on 04/13/13 ~11:00 AM Pacific
Frequency of dropped tweets increased starting on 4/17/13

Past 24 hours: 5.3% of tweets were not received via streaming API
Since 4/13/13: 3.78% of tweets were not received via streaming API

No limit messages were received while streaming, so the dropped tweets do not appear to be intentional.

Thanks

FrediAkai · 2013-05-01T02:33:09.000Z

It does happen to me as well; some tweets in the timeline and mentions are missed. And it happens on every single application I’ve tried (TweetDeck, TweetBot, Echofon and Twitter for Mac).

I consider it to be a very bad issue, so I’d be thankful if you could fix it in the following days.

lgladdy · 2013-05-01T10:22:27.000Z

It took me a while to confirm this, but i can now say with confidence this also affects timeline tweets, not just your own tweets and replies to you.

That is to say just any tweet from any person you follow might not be shown in your timeline too. Based on @dcftt’s research, this suggests that if you use a twitter streaming client you’ll be missing 5.3% of all tweets, which is a pretty serious bug!

dcftt · 2013-05-07T15:59:09.000Z

It appears that this issue was effectively resolved last week on May 1st. I am still completing some tests, but things are looking good! Thanks again

mastipabs · 2013-07-01T06:48:14.000Z

@dcftt, you confirmed the issue was effectively resolved, We are still seeing this issue using 1.1 streaming api, no rate limit notifications, but missing close to 10% of data for the track words.
@kurrik can you please confirm if any fix has been put in for this. I see the issue
https://dev.twitter.com/issues/999 in opened state. Thanks!

robinrowell · 2014-06-16T21:21:05.000Z

I am having the same issue where tweets are missing – was a fix ever found?

muteesocrates · 2014-09-22T12:56:20.643Z

I don’t know if this issue was marked as resolved but the streaming API doesn’t seem to capture tweets with self user mentions.

_clopez · 2017-10-19T23:02:24.587Z

We are still having the same issue.

We have missing mentions and dm’s targeted to our managed accounts and even our own responses using the Stream API. The report of missing tweets has increased on the past two weeks.

An official response by the Twitter Team that the stream doesn’t guarantee the 100% of tweets would be much appreciated, so our customers could understand the situation.

NathanMEvans · 2017-10-31T10:36:38.297Z

We have started to get reports of missing DMs from our users when using the streaming API. Is there a fix for this?

andypiper · 2017-10-31T11:21:10.479Z

I’m not aware of any specific issues with the existing API endpoints at the moment. Are these high volume users?

If you’re using user streams, we’d suggest that you start to look into the Account Activity API which will ultimately replace them. Currently Direct Message events are available, and more events will be available at GA. Additionally, if you are building on the Direct Message endpoints for posting, we would suggest that you consider the new beta features and endpoints for greater future proofing.

_clopez · 2017-10-31T12:26:58.713Z

Hi Andy, thanks for your answer. It happens quite often, but it’s like 1 in 1000 tweets/DM that our customers report that are missing (we handle big companies Twitter accounts). We have already requested access to Account Activity API Beta, but no response yet, is there any priority or criteria for beta access? Like Country or number of applications?