Using streaming api to get brand tweets - sample of tweets?

streaming
api

#1

HI!
I am building an app and currently using the streaming api to get all tweets and interactions for brands, example, I Connect Mc Donalds and get all tweets from them.

I want to know if using the streaming api I have any chance of losing tweets because of the ‘sample’:

The following streams offer samples of the public data flowing through Twitter. Once applications establish a connection to a streaming endpoint, they are delivered a feed of Tweets, without needing to worry about polling or REST API rate limits.

Many thanks in advance!!!


#2

This will depend on the endpoint you are using, and the volume of interactions you are seeing.

  • statuses/sample is always just a 1% sampling of the main firehose. I assume you’re not using that one, as you’re talking about getting Tweets about a specific brand.
  • statuses/filter using e.g. the track parameter is also limited to an overall 1% volume cap. This means that if the term you are tracking is at a particularly high volume “spike” for some reasons, there’s a chance that you may miss some Tweets. However, in most cases this is not a practical limitation as the Tweet volumes on any topic do not often jump to a level where this is a problem.

If you absolutely require comprehensive coverage, then you’d want to use the commercial PowerTrack API in the future as your solution scales.

Lastly, just to point your attention to our recent announcement about the future deprecation of user and site streams. We expect the other sample / filter APIs to continue to be freely available, in a more robust form, in the future. We’ll be making more streaming data available at new commercial tiers along the road as well.


#3

Hi Andy!

There is something that I’m not really sure I’ve understood.

When you say that is limited to a overall of 1%:

  • Does it means that if my search/filter override the 1% of the tweets going on in that period, perhaps I don’t get all the matches.
    or …
  • Does it means that the search and filter is done within 1% of the total tweets going on in that period

or something else? ha!

Anyway, just wondering, did it happen that some hashtag rised the 1% of tweets ever?

Thanks and Regards,
Rodrigo.


#4

If you are using the filter endpoint today, then if your tracking term happens to exceed 1% of the firehose full volume then you may receive notices that you are dropping results.

The filter is not done within the 1% sample. It is 1% of the total.