Accessing media from protected tweets

[timkindberg]

I perform a hashtag-based search through the API for videos. This only finds unprotected tweets. Now suppose the author of such a tweet subsequently makes her account protected. I’ve discovered that, by and large, the video (and associated thumbnail) URLs I’ve retrieved still work, i.e. I can retrieve the video and its thumbnail. But not always: sometimes I get a 404. The information I’ve found online https://support.twitter.com/articles/14016 says:

“If your Tweets are protected, only your followers will be able to see your media in your Tweets. Please note that your followers may download or re-share links to media that you share in protected Tweets. Links to media shared on Twitter are not protected. Anyone with the link will be able to view the content.”

• which suggests to me that the media URLs I’ve retrieved should still work.

So why am I randomly experiencing 404s?

Thanks for any help.

[timkindberg]

I would appreciate a response from Twitter to this question. The use model respects the owners of protected tweets: essentially, they make something (a CGI construction) from their own tweeted content, but may later decide to hide the tweets themselves. I.e. it’s not to grab the content from third parties and then expose it even though they’ve subsequently protected their account. And if anyone deletes a tweet, the content is no longer shown.

[andypiper]

Thanks for the question - I’m unable to confirm the expected behaviour here at the moment as I’ve been unable to investigate so far.

[andypiper]

Having looked at this a little this morning, I would speculate that there may be cases where an account becomes protected, and then subsequently the user deletes the Tweet that had media content attached - but I can’t be sure that is what you’re encountering.

In general though, you should not be storing Twitter data (beyond Tweet IDs) longer term, so the cases where you’d end up going back and getting a 404 on a media URL should be pretty rare.

If the originating Tweet has become unavailable to your app for some reason - either due to deletion or protection - then you should not be surfacing the linked content, per the developer policy sections on maintaining integrity of Twitter’s products and respecting users’ privacy.

[timkindberg]

Thanks Andy,

I guess another way of putting it is this: if a user deletes her tweet, I will find out through a 404 when I attempt to retrieve the associated content e.g. image. But if a user protects her account and hence her tweet, I would not expect a 404 since others (her followers) can still retrieve it and the media URL has (presumably) not changed. So something else must be going on in these particular cases - I’ve been using test accounts and know the tweet has not been deleted. I’ve found no particular pattern, except a few cases of the 404 going away if I change an account back to unprotected.

What is the recommended way to tell whether a user’s account has become protected?

[andypiper]

The user object has a field protected that is set to true if the account is protected. That would be the best way to check.

[andypiper]

Thinking about this some more - and to be clear, the earlier comments about protected status and not surfacing content from protected accounts unless there is a follow relationship still stand - I suspect this could also be related to CDNs. It can take a certain amount of time for different content caches to synchronise in different networks. So this could be an eventual consistency case.

[timkindberg]

Thanks Andy,

I guess you’re saying that all CDN nodes will eventually give a 404 for media URLs obtained from a hashtag search, where the account was subsequently protected. Thinking about it, I never seem to get 404s for media URLs acquired from reading a protected user’s timeline as a follower. I guess the URLs that Twitter hands out encode their genesis in that respect, so that you can have them produce a 404 if the account becomes protected but only in the hashtag case.

[andypiper]

I think the only circumstance I can see this happening is where you’ve previously run that search and stored the results and then go back some time later to pull the media. Protected accounts should not show up in search.

[timkindberg]

Yes, I can afford to be lazy in the case where I pull from a protected account’s timeline in the first place. I hadn’t thought through the differences when I use the results from a previous search.