Is http://urls.api.twitter.com/1/urls/count.json? a valid way to get the tweet count for a URL?

Tidwell · 2012-02-09T20:22:28.000Z

My question is about this URL: http://urls.api.twitter.com/1/urls/count.json

The above url is clearly used internally in the tweet button, and does return valid data when used with a jsonp request from any domain - but is it permissible to have the client’s browser directly make a jsonp requests to this URL (via jquery or just a standard script tag)?

In my case, I’m trying to create a consistant look for our social buttons, but still need to get counts in some way. I see this URL bandied about all over, but can’t find any mention of it in Twitter’s API docs, and want to know if we would run into trouble if I used it on our (very highly-trafficked) site.

episod · 2012-02-09T21:04:34.000Z

Hi there,

Sorry, we don’t allow usage of that endpoint in any other contexts than the Tweet Button. If the counts are important to display on your site, you’ll either need to use the official Tweet Button or count the tweets yourself using the Streaming API and provide your own counts.

jonjhiggins · 2012-02-10T11:14:00.000Z

I can’t figure out how you would do this with the streaming API, is there any documentation / examples anywhere on returning a count of URL mentions?

It seems like a popular requirement - but when Googling this issue everyone suggests using the http://urls.api.twitter.com/1/urls/count.json method that’s unsupported by Twitter. The only mention I could find of using the streaming API to do this was someone also at a dead-end (http://stackoverflow.com/questions/6730249/how-to-track-urls-using-twitter-streaming-api)

The only alternative I have found is to use the Topsy Otter API, e.g http://otter.topsy.com/urlinfo.json?url=

Thanks

episod · 2012-02-10T15:54:15.000Z

The streaming API tracks strings, and URLs are just a kind of string. The statuses/filter.json method of the streaming API lets you supply a “track” parameter with comma-separated strings that you’re matching for on the streaming API. If you URL-encode a URL as one of those strings and start listening to the stream, you’ll be streamed a tweet every time that URL is tweeted or retweeted (provided that tweets containing that URL make up no more than 1% of all tweets happening at that moment – in all likelihood not an issue for a single URL). As each tweet comes in, you count them. You won’t be streamed deletes or unretweets this way, but it’s enough to get started with.

A single-URL example:
curl -u user:password “https://stream.twitter.com/1/statuses/filter.json” -d “track=https%3A%2F%2Fdev.twitter.com%2Fdiscussions%2F5653”

{ "favorited":false, "text":"Testing link https:\/\/t.co\/jvivDycK", "possibly_sensitive_editable":true, "truncated":false, "created_at":"Fri Feb 10 15:51:21 +0000 2012", "retweeted":false, "retweet_count":0, "coordinates":null, "source":"\u003Ca href=\"http:\/\/realitytechnicians.com\" rel=\"nofollow\"\u003EOAuth Dancer Reborn\u003C\/a\u003E", "in_reply_to_status_id_str":null, "possibly_sensitive":false, "entities": { "user_mentions": [ ], "urls": [ { "indices": [ 13, 34 ], "url":"https:\/\/t.co\/jvivDycK", "display_url":"dev.twitter.com\/discussions\/56\u2026", "expanded_url":"https:\/\/dev.twitter.com\/discussions\/5653" } ], "hashtags": [ ] }, "geo":null, "in_reply_to_user_id_str":null, "place":null, "in_reply_to_user_id":null, "in_reply_to_screen_name":null, "id_str":"167999101609320448", "user": { "default_profile":false, "notifications":null, "profile_use_background_image":true, "time_zone":null, "created_at":"Wed Mar 03 19:37:35 +0000 2010", "verified":false, "geo_enabled":true, "profile_text_color":"333333", "profile_background_image_url":"http:\/\/a0.twimg.com\/profile_background_images\/80151733\/oauth-dance.png", "description":"", "default_profile_image":false, "profile_link_color":"0084B4", "url":"http:\/\/bit.ly\/oauth-dancer", "follow_request_sent":null, "favourites_count":6, "lang":"en", "profile_background_image_url_https":"https:\/\/si0.twimg.com\/profile_background_images\/80151733\/oauth-dance.png", "profile_background_color":"C0DEED", "followers_count":23, "profile_image_url":"http:\/\/a2.twimg.com\/profile_images\/730275945\/oauth-dancer_normal.jpg", "contributors_enabled":false, "profile_background_tile":true, "protected":false, "profile_image_url_https":"https:\/\/si0.twimg.com\/profile_images\/730275945\/oauth-dancer_normal.jpg", "location":"San Francisco, CA", "profile_sidebar_fill_color":"DDEEF6", "name":"OAuth Dancer", "id_str":"119476949", "listed_count":0, "following":null, "screen_name":"oauth_dancer", "id":119476949, "is_translator":false, "show_all_inline_media":false, "statuses_count":153, "utc_offset":null, "friends_count":14, "profile_sidebar_border_color":"C0DEED" }, "id":167999101609320448, "contributors":null, "in_reply_to_status_id":null }

jonjhiggins · 2012-02-11T11:22:52.000Z

Brilliant, thank you for the example

tonariman · 2012-03-06T14:25:14.000Z

Hi Taylor;

I’m trying to understand the rationale behind this recommendation.

What should be a relatively simple problem has been flipped on its head and turned into a massively complicated undertaking. For apps that track hundreds/thousands of accounts, it’s not practical to use the Streaming API (limited predicates) to track URL tweets and retweets, you’d only be able to measure 400 at a time. If you processed in batches, you’d have gaps in data/accuracy. Site Streams + Control Streams is overkill and would only include mentions of the URL by users or their followers. The only solution that works reliably is if you sent the aggregate count alongside the URL in the Streaming API response, as is done with the retweet_count.

This is a number that twitter has already processed… why not expose it? Why force every client to redo the work? This imposes a drastic toll (storage, processing, connectivity complexity) on not only the client but on twitter and the network too. Where’s the benefit?

Regards,
N.

tkeyling · 2012-03-06T00:10:09.000Z

@tonariman Good point! I don´t get that, too. Another point regarding the reliability of the tweet-count is, that some domains are systematically ignored, f.e. almost all NYTimes-Articles. For those, you can querry the tweets from topsy (on the other hand, topsy’s stats won´t show tweets for the most cnn.com articles, but Twitter API does). Is there an somehow official constraint for some urls like nyt?
Greetings,
T.

claytonbolz · 2012-03-22T20:17:25.000Z

We also need this feature available in the APIs. We do a similar thing to collect the Facebook like counts and Facebook provides a simple REST call we can do, plus batch 20-50 URLs in the HTTP request which reduces a lot of network traffic and load. Twitter definitely needs this as well.

It doesn’t make sense to stream each URL and manually collect the retweets, it’s not scalable. Is there a place were we can add this as a feature request?

Joan_Artes · 2012-04-04T12:28:37.000Z

And we can use cdn.api.twitter.com/1/urls/count.json to retrieve the count? Like this way -> http://joelb.me/blog/2012/tutorial-creating-a-custom-tweet-button-with-counter/

episod · 2012-04-04T14:34:42.000Z

It’s a known desire that you want this data. At this time, usage of the http://urls.api.twitter.com/1/urls/count.json API directly is s
till forbidden. The only valid source of this data for third parties is if you derive it yourself at this time. You should just consider it data unavailable to you at this time if the means available to you is unscalable for you to track.

sara_mose · 2012-04-04T15:15:13.000Z

So you are saying that any developer who uses the API directly on a users account, without their knowledge and does things like change passwords and direct all their personal phone calls ans text messages through google voice, and intercepts all their emails before sending some - but perhaps not all - to the intended recipient using google, twitter and Facebook development tools, are you saying that the developer knows that what they are doing is wrong? Because that’s pretty much how I feel, being the recipient of this treatment, I find it fascinating to hear it said that developers know they are doing something very wrong. Interesting topic.

sara_mose · 2012-04-04T15:19:49.000Z

And as a programmer myself who did batch development fo hundreds of thousands of customers, your answer makes perfect sense. It’s completely illogical for what the tool was designed for. Look at the problem differently though, if you were not interested in hundreds or thousands of users, but just a very few, doesn’t the question now make sense?

episod · 2012-04-04T15:19:49.000Z

I don’t follow what you’re saying or the relation of that to an undocumented endpoint?

sara_mose · 2012-04-04T15:23:58.000Z

For the record the answer to the question, or at least how it was answered just now for me personally, is simply knock the user out of Twitter and lock them out until you get what you want.

Joan_Artes · 2012-04-04T15:24:00.000Z

I’m using the API that uses the your widget.js: http://cdn.api.twitter.com/1/urls/count.json

What’s the difference between http://cdn.api.twitter.com/1/urls/count.json and http://urls.api.twitter.com/1/urls/count.json ?

sara_mose · 2012-04-04T15:27:14.000Z

You probably don’t understand because I’m not a twitter developer, so the better question is, why is my personal twitter ID able to get into this development site? It’s what I have been asking myself. Enough on this topic, though, don’t want to take up too much of your time on my first time here. Many thanks.
mosey78@gmail.com

episod · 2012-04-04T15:41:10.000Z

They are effectively one and the same.

episod · 2012-04-04T15:42:14.000Z

Twitter developers use their own Twitter logins and passwords to log into this site and discuss the API & platform. If you want to cease getting any emails from this developer’s site, you can address your settings at [alias:/user].

Joan_Artes · 2012-04-04T15:52:20.000Z

Ok. Thanks.

sara_mose · 2012-04-04T15:56:19.000Z

So you really aren’t concerned about the programming being developed and approved here - by you- and what you are aware of right now that should want you to protect your development rights, you are really just not wanting me to be inconvenienced? Gotcha. I’m glad everything is so well documented, very nice coding standards as far as that goes. You have my email. Enough said.