Streaming API missing some timeline, replies and own tweets


#1

Since probably last Friday or Saturday, the streaming API has randomly be been missing some @replies to me, and when I post my own tweets they sometimes don’t appear back in the streaming tweet, meaning some clients like tweetbot don’t show it.

A twitter search for “twitter streaming missing” suggests i’m not the only one affected by this.

Has anyone else seen this issue?


#2

Could you provide more information here? Specifically I’m looking for a reproducible way to recreate your problem.


#3

I’ve been trying to find a reliable reproduction method here but haven’t found one. It’s possible it excludes some normal tweets too, but obviously that’s harder to know (I only notice when my own tweets don’t come back in the user stream and when I notice replies on my mentions tab when I scroll back through).

I had a bunch of replies on my tweet asking if anyone else had seen this, so it seems to be pretty common. I’m happy to provide information or tweet_ids that were missing, but I guess that won’t help you very much, as the tweets do still appear fine outside of streams.


#4

Is your test just based off of using Tweetbot or actually consuming the API directly? My understanding is that Tweetbot will transition off of user streams if the connection is spotty, for example.

You can send me (DM if you want) IDs of Tweets which were affected but unfortunately there’s not much we can do after the fact.

If anyone has a case where they’re able to connect directly to userstreams (via curl or something like that), Tweet, and then not have that Tweet show up in the stream, I would be very interested in investigating that.


#5

My instances have been Tweetbot based, but the replies I got on twitter were affecting Echofon too. I’ll keep a curl stream open and try and reproduce that issue now though!


#6

Hey Arne,

I’ve been working on reproducing this. It does seem to be genuinely random. I sent 10 tweets from my test development account which is private (@gmeddev) to me (@lgladdy). The first 9 arrived in both a curl client, and my tweetbot client. The 10th didn’t arrive in either - but it does show in my mentions tab of twitter.com

That 10th tweet is https://twitter.com/gmeddev/status/327490068690370562

I can make the account unprotected if you need it to be. The protected part doesn’t matter.


#7

Hmm. weirdly the comment I posted yesterday still hasn’t made it through moderation…

I have been able to confirm this issue with a curl connection to a userstream. It took about 13 tweets from my developer account to myself to have one of them missing from my stream.


#8

I’ve done some more research this morning into this… This time it took 23 tweets from one of my dev accounts to my main account for 1 to be missing.

I had 3 userstream connections open, and the tweet was missing on all of them.

Hopefully you guys at twitter HQ should be able to reproduce this too! I’m happy to give you any more information I can, or tweet IDs that were missing to see if its just a weird issue that maybe one or two of your servers aren’t pushing tweets routed through them correctly.


#9

Thanks for doing the testing. I’ll look into it more thoroughly. Is there any pattern to the Tweets which did not go through? What was the test text you were using?


#10

Nothing unusual about it,

I @'d myself and added an incremental integer to the end of each one.

Yesterday’s test was with: “@lgladdy This is a test of a normal-ish tweet :slight_smile: $int” and today’s was: “This is a test tweet to see how many it takes for one to be excluded from the userstream: #$int”


#11

In the past few days, I have been noticing that tweets have not been making it through the streaming API, seemingly on a random basis. I have not been receiving any limit messages, so I don’t think the dropped tweets are intentional.

I will also add that I’ve been consuming the streaming API for a few years and this seems to be the first time I have experienced the issue of tweets getting dropped. I haven’t made any software changes recently. Please let me know if there is any additional info I can provide and thanks for looking into things!


#12

Hm, I tried to reproduce your results using the same approach but haven’t had any success. Could you give me links to the specific Tweets which were not streamed?


#13

Both:

https://twitter.com/gmeddev/status/327694357211799552
and
https://twitter.com/gmeddev/status/327490068690370562

Are tweets i’ve sent which haven’t arrived in the steams. Also this tweet from a friend at me didn’t come through:


#14

If you take a look at @gmeddev’s feed I just wrote some code to send tweets to help me demonstrate this to you.

Tweets #11 and #15 didn’t appear in my userstream.


#15

Thanks, that’s helpful.


#16

Not sure if you’ve fixed anything, but right now this issue isn’t happening. I’ve written a twitter app that does nothing but prove the existence (or not) of this bug…

It opens a userstream connection, then sends 25 tweets and makes sure those tweets appear in the feed.

So either you’ve fixed it, or this only happens at certain times, and isn’t a consistent thing. I’ll keep an eye on it, and if i notice it happening again I’ll drop you a tweet with a link to my app that shows you all the IDs. :slight_smile:


#17

I think it may be an intermittent thing. I’ll be testing at various times with a similar approach, thanks.


#18

We also experienced this issue two days ago and about about a week ago. Some @replies and own tweets were missing from the user stream.

We noticed this issue after switching to v1.1 of user stream recently.


#19

Also experiencing this sporadic omission of tweets while using Tweetbot and after discussing it with the author he sent me here since it seems to be the API itself…

Specific examples: https://twitter.com/jvanegue/status/328941490552311808 (private) he mentions me, I follow him, and it doesn’t show up

https://twitter.com/0xabad1dea/status/328930161854656512 my own tweet, never shows up in timeline, yet is visible on profile

Both show up as expected in official iPhone client.

I’m a heavy tweeter with a LOT of replies, may or may not be relevant


#20

Happened to me now for two weeks aaaaaa. And just happened right now saw the tweet roll over at top of screen but not in my mentions …please help fix it …