How should I handle a mobile website redirect with Twitter cards?

dblockdotorg · 2013-09-16T19:32:58.000Z

Dear Birdie,

We have http://m.artsy.net and http://artsy.net, two separate websites. The first one handles mobile clients and redirect browsers.

We have users sharing the m. website, like so: https://twitter.com/theothernight/status/379669293098864640. We want that to automatically become the non-mobile website so that the card appears.

Is it enough to detect the Twitter BOT (and if so what do we look in for the user agent)? What’s the best way to handle this?

rchoi · 2013-09-25T16:08:42.000Z

Hi Daniel,

One option is to have the Card appear on both properties.

Otherwise, a redirect should tell our Twitterbot to cache the Card for the final destination accordingly.

Ryan

dblockdotorg · 2013-09-25T16:16:48.000Z

Ryan, I cannot find any documentation on what Twitterbot user-agent is? Anything?

dblockdotorg · 2013-09-25T17:27:00.000Z

Nm, found it here: https://dev.twitter.com/docs/cards/getting-started, Twitterbot.

dblockdotorg · 2013-09-27T22:57:00.000Z

The validator tool doesn’t test these.

Artsy – 26 Sep 13

The Artsy Questionnaire: Raul de Molina, Photojournalist-Turned-Collector and...

We’re always excited to find a collector with an interesting outlook, and Raul de Molina certainly fits the bill. When he’s not winning Emmys for hos...

Validator broken, but redirect is there:

$ curl -v http://m.artsy.net/post/editorial-the-artsy-questionnaire-raul-de-molina-photojournalist-turned-collector -H “User-Agent:Twitterbot”

About to connect() to m.artsy.net port 80 (#0)
Trying 54.243.58.214… connected
Connected to m.artsy.net (54.243.58.214) port 80 (#0)

GET /post/editorial-the-artsy-questionnaire-raul-de-molina-photojournalist-turned-collector HTTP/1.1
Host: m.artsy.net
Accept: /
User-Agent:Twitterbot

< HTTP/1.1 302 Moved Temporarily
< Content-Type: text/plain
< Date: Fri, 27 Sep 2013 22:57:09 GMT
< Location: http://artsy.net/post/editorial-the-artsy-questionnaire-raul-de-molina-photojournalist-turned-collector
< Vary: Accept
< X-Powered-By: Express
< Content-Length: 137
< Connection: keep-alive
<

Connection #0 to host m.artsy.net left intact
Closing connection #0

dblockdotorg · 2013-10-01T14:43:41.000Z

And just to be clear, the tweets with m.artsy.net don’t show cards, the redirect has been in place for a week.

rchoi · 2013-10-01T16:36:38.000Z

Thanks for the detail. This is super helpful.

The only thing I can think of is changing the 302 to a 301. I looked at doing a curl against a twitter URL and saw this:

Connected to t.co (199.59.150.44) port 80 (#0)

GET /RAFOoJrOD2 HTTP/1.1
Host: t.co
Accept: /
User-Agent:TwitterBot

< HTTP/1.1 301 Moved Permanently
< cache-control: private,max-age=300
< date: Tue, 01 Oct 2013 16:35:06 GMT
< expires: Thu, 01-Jan-1970 00:00:00 GMT
< location: http://cardstest2.blogspot.com/2013/09/barnotes2.html
< server: tfe
< set-cookie: muc=1;Domain=t.co;Expires=Sun, 13-Sep-2015 16:35:06 GMT
< Content-Length: 0

dblockdotorg · 2013-10-11T18:38:33.000Z

Changed it to a 301, no change.

The validator is still broken and doesn’t follow links.
Twitter bot doesn’t seem to be following it either, see https://twitter.com/dblocktest/status/388735110536630272 vs. https://twitter.com/dblocktest/status/388735133433348097

Can someone with their hands in the code please take a look? I’ve been tracking this for almost a month.

rchoi · 2013-10-12T02:06:10.000Z

Sorry for the delay. Been very swamped.

Out of curiosity, could you try to do a test to see what t.co and bit.ly use as redirects (since they both work), and see how that might differ from what you’re doing?

Please post the response here.

Thanks!

dblockdotorg · 2013-10-14T13:51:19.000Z

Ok, figured this out. The user-agent that Twitter sends is Twitterbot/1.0, not just Twitterbot. I should have checked the logs. We just needed to match that.

This took way too long to resolve for something that silly. If I may suggest a few things for the Twitter team:

Document that twitter cards support redirects. Figure out whether a permanent vs. temporary redirect is actually required so that we’re not left guessing.
Document the twitterbot UA fully, maybe in a single FAQ, whenever something changes (version, etc.), document that too.

Thanks,
-dB.

rchoi · 2013-10-14T16:50:52.000Z

Apologies, and hope looking at bit.ly helped.

I’ve updated a couple places of documentation to specifically call out the version as well:

https://dev.twitter.com/docs/cards/getting-started
https://dev.twitter.com/docs/cards/troubleshooting

Know it’s too late for you, but hope it helps others.

Let me know if there are other places this would be useful.

dblockdotorg · 2013-10-14T17:25:44.000Z

Add something about the fact that Twitter follows redirects. Ideally figure out whether 301 or 302 or both work. Also how far does Twitter follow redirects? Till forever unless there’s a loop?

dblockdotorg · 2013-10-15T15:43:03.000Z

@rchoi, another problem is SSL, the validator says https://artsy.net/artwork/julian-opie-woman-in-the-rain-with-hat-1 is OK, but https://twitter.com/dblocktest/status/390139893138989056 doesn’t show a card.

Two things that are different from the mobile site problem:

This is a 301 non-permanent redirect.
This site has a robots.txt that disallows robots. But it’s a redirect so I assume it shouldn’t matter at all.

Given that the validator works, could you please provide some definite answers instead of me poking in the dark?

dblockdotorg · 2013-10-16T00:47:05.000Z

@rchoi, update, this just started working, I suppose something was cached (or someone fixed something), even though it’s weird as I was trying new URLs