TwitterBot


#1

I have updated the robots.txt in our web app (digitalnz.org) to get twitter card working. It is working for one path but not other.

robots.txt

User-agent: Twitterbot
Disallow: *
Allow: /records/* (Working)
Allow: /user_sets/* (Not Working)

User-agent: *
Allow: /
Disallow: /records
Allow: /records/*
Allow: /user_sets/*
Crawl-delay: 1

In card validator i am getting this message for user_sets

INFO: Page fetched successfully
INFO: 5 metatags were found
WARN: Not whitelisted


#2

What is the exact URL you are validating?


#3

I tried this urls in the validator.

And these are the meta we have on this page.

`

`

#4

When I retrieve that page using curl -A Twitterbot http://www.digitalnz.org/user_sets/503a96141257577392004283 I don’t see the meta tags. Are you injecting them via Javascript after the page loads? That will not work.


#5

Thanks @andypiper. We will have a look at this and get back to you if needed.


#6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.