TwitterBot

EdwinARozario · 2016-11-03T02:26:45.070Z

I have updated the robots.txt in our web app (digitalnz.org) to get twitter card working. It is working for one path but not other.

robots.txt

User-agent: Twitterbot
Disallow: *
Allow: /records/* (Working)
Allow: /user_sets/* (Not Working)

User-agent: *
Allow: /
Disallow: /records
Allow: /records/*
Allow: /user_sets/*
Crawl-delay: 1

In card validator i am getting this message for user_sets

INFO: Page fetched successfully
INFO: 5 metatags were found
WARN: Not whitelisted

andypiper · 2016-11-03T09:20:37.988Z

What is the exact URL you are validating?

EdwinARozario · 2016-11-03T20:11:44.899Z

I tried this urls in the validator.

DigitalNZ

Surf Lifesaving

A DigitalNZ story by: Christine Nana -

And these are the meta we have on this page.

`

andypiper · 2016-11-04T13:55:17.840Z

When I retrieve that page using curl -A Twitterbot http://www.digitalnz.org/user_sets/503a96141257577392004283 I don’t see the meta tags. Are you injecting them via Javascript after the page loads? That will not work.

EdwinARozario · 2016-11-06T20:07:04.694Z

Thanks @andypiper. We will have a look at this and get back to you if needed.

system · 2016-11-20T20:07:13.789Z

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.