Stream API Documentation Issues & Bugs

[JamesMcMinn]

The new documentation seems to be considerably better than the old version (well done to all involved and congratulations on the launch), however I’ve come across a few issues:

https://developer.twitter.com/en/docs/tutorials/consuming-streaming-data#

• Broken image at the top of page. Image seem to work if I load it directly but embedded version won’t load. Console gives the following error: "Refused to load the image ‘http://support.gnip.com/img/powertrack_flow.png’ because it violates the following Content Security Policy directive: “img-src ‘self’”

https://developer.twitter.com/en/docs/tweets/filter-realtime/api-reference/post-statuses-filter

• Links to documentation for each of the paramaters is missing. “See track for more infromation” leads to https://developer.twitter.com/en/docs/tutorials/consuming-streaming-data#track which has no information about the track keyword. I can’t seem to locate the information anywhere?
• Example request is…lacking?

[andypiper]

Thanks for the feedback! I’ve got an open ticket internally that covers some of these, but not all of them. We’ll take a look at resolving asap.

[abstractitsthlm]

Is there any update on this issue?
I am desperately looking for the details on the parameters but cannot find it.

[andypiper]

Which parameters are you looking for? track?

[abstractitsthlm]

yes, track.

[andypiper]

track

A comma-separated list of phrases which will be used to determine what Tweets will be delivered on the stream. A phrase may be one or more terms separated by spaces, and a phrase will match if all of the terms in the phrase are present in the Tweet, regardless of order and ignoring case. By this model, you can think of commas as logical ORs, while spaces are equivalent to logical ANDs (e.g. ‘the twitter’ is the AND twitter, and ‘the,twitter’ is the OR twitter).

The text of the Tweet and some entity fields are considered for matches. Specifically, the text attribute of the Tweet, expanded_url and display_url for links and media, text for hashtags, and screen_name for user mentions are checked for matches.

Each phrase must be between 1 and 60 bytes, inclusive.

Exact matching of phrases (equivalent to quoted phrases in most search engines) is not supported.

Punctuation and special characters will be considered part of the term they are adjacent to. In this sense, “hello.” is a different track term than “hello”. However, matches will ignore punctuation present in the Tweet. So “hello” will match both “hello world” and “my brother says hello.” Note that punctuation is not considered to be part of a #hashtag or @mention, so a track term containing punctuation will not match either #hashtags or @mentions.

UTF-8 characters will match exactly, even in cases where an “equivalent” ASCII character exists. For example, “touché” will not match a Tweet containing “touche”.

Non-space separated languages, such as CJK are currently unsupported.

URLs are considered words for the purposes of matches which means that the entire domain and path must be included in the track query for a Tweet containing an URL to match. Note that display_url does not contain a protocol, so this is not required to perform a match.

Twitter currently canonicalizes the domain “www.example.com” to “example.com” before the match is performed, so omit the “www” from URL track terms.

Finally, to address a common use case where you may want to track all mentions of a particular domain name (i.e., regardless of subdomain or path), you should use “example com” as the track parameter for “example.com” (notice the lack of period between “example” and “com” in the track parameter). This will be over-inclusive, so make sure to do additional pattern-matching in your code. See the table below for more examples related to this issue.

Track examples:

Parameter value	Will match...	Will not match...
Twitter	TWITTER twitter “Twitter” twitter. #twitter @twitter http://twitter.com	TwitterTracker #newtwitter
Twitter’s	I like Twitter’s new design	Someday I’d like to visit @Twitter’s office
twitter api,twitter streaming	The Twitter API is awesome The twitter streaming service is fast Twitter has a streaming API	I’m new to Twitter
example.com	Someday I will visit example.com	There is no example.com/foobarbaz
example.com/foobarbaz	example.com/foobarbaz www.example.com/foobarbaz	example.com
www.example.com/foobarbaz		www.example.com/foobarbaz
example com	example.com www.example.com foo.example.com foo.example.com/bar I hope my startup isn’t merely another example of a dot com boom!

[andypiper]

You should not build any long-term dependency on this endpoint or these parameters as they are subject to replacement in the future.

[abstractitsthlm]

Thank you!!

Ah ok… is there something that I can use for long term already in place available today. If so what would that be?

[andypiper]

For streaming just on the filter endpoint you’re good, but user streams are due to be replaced by the Account Activity API in the coming year or so. I’d suggest keeping an eye on our roadmap for a sense of what is happening, and these forums for announcements related to endpoint changes in the future.

[amanodev69]

Hi,

Just to be clear on the subject : the streaming endpoint statuses/filter isn’t going to be deprecated, for both track and follow parameters, right ?
The ones that will no longer be supported are the User and Site Stream (that are no longer listed in the new documentation by the way, but still mentionned in the guide section) in favor of the (still in beta) Account Activity API ?

Currently, the documentation is very unclear on the stream subject especially with the Gnip notion. For instance, when you click on “Contact Gnip” you are redirected to https://developer.twitter.com/en/docs, which doesn’t give any information on how you can contact them.

I’m glad I’m not diving now in the streaming thing, as it’s nearly impossible to grasp the way to implement it without Gnip (that is clearly highlighted in comparison to the free statuses/filter way) with this incomplete documentation.

BR,
Simon

[andypiper]

We’re very sorry for the confusion here, and we will work to make this much more clear, as the last thing we want is to put folks off from building solutions with us.

We’ve said we plan to replace statuses/filter and statuses/sample) on the roadmap, but not remove them. Hopefully that process will result in better reliability and data quality, however I anticipate that it will require code rework in the medium term, as some parameters and formats (not overall functionalitis) are likely to change.

That contact link is broken indeed, and I’ll add that to my bug tracker to get fixed.