Hi guys,
Just going to chip in a little note on this: It’s not a realistic prospect right now for crawlers in general—including ours—to start executing JavaScript for meta tags. In terms of performance both of the page execution, and initialization of the target URL (requesting a single HTML resources in one request, verses a plethora of JavaScript dependencies before the page can be executed, let alone the content itself be downloaded and put into place for extraction.) Remember, we’re trying to get this metadata back from your site and into a Tweet with minimal delay as that Tweet is fanned out to thousands of recipients in the milliseconds after you send it. The only way this is going to work in a performant manner is by parsing indexable HTML.
To Con’s point: The previous iteration of our own site was rendered entirely on the client, and honestly we suffered similar shortcomings: It’s only since we reengineered that features for crawler—such as OEmbed autodiscovery for embedded Tweets—can be added to permalink pages. Before we’d have suffered the same problem with CMS systems needing to download and execute scripts to get the context. We’re now using a system of initial rendering on the server, and partial page loads as you browser around the site. (We blogged about the architecture change back May: http://engineering.twitter.com/2012/05/improving-performance-on-twittercom.html)
Ben
