It seems like Twitter caches robots.txt settings, or something like that. Whatever I do, previewing my cards always shows the following error:
“Pre-fetching image http://creativeskills.be/cache/flickrjobs/visual-designer-xaop-13-11-12_25.png failed because it’s denied by robots.txt”
While my robots.txt file is exactly this:
User-agent: Twitterbot Disallow:/cache/ Allow: /cache/images/ User-agent: * Disallow:/cache/ Allow: /cache/images/
You can check the robots.txt file here: http://creativeskills.be/robots.txt
Even when I empty the robots.txt file I get this error. But the weird thing is, sometimes (let’s say in 5% of the cases) the preview successfully shows the card, but when trying again (with the same URL and without making any changes to robots.xt) I get the above error.
So it looks like the Twitter Preview tool is unreliable or just moody.