We have absolute sure that our website allow robots.txt to verify the url: http://rfbd.cm/rp88d507b8
However, twitter said denied by rotbots.txt…
But in last week, it works…
Can anyone or twitter to check what happens?
The error message I’m seeing is that your site doesn’t have any cards markup / there are no metatags.
Hi Andypiper,
what I saw is this
Weird, because I’m seeing:
INFO: Page fetched successfully WARN: No metatags found WARN: this card is redirected to https://www.referboard.com/rp88d507b8
Saying that, your robots.txt file is disallowing all robots from accessing / which thus overrides all of the other rules in your file.
we try following rules, is that can be compatible with twitter?
Allow: /rp* Allow: /robots.txt Disallow: /dev/ Disallow: /
The last line Disallow: / basically says, don’t allow any robots to access anything below / which is the root level of your site. So that stops any of the other rules from having any effect at all.
Disallow: /
/
ok, but why other robot engine can accept ours?
We have already get rid of “Disallow: /” to be particular directories.
How long does twitter will re-cache the robots.txt?
We don’t cache the robots file. I just checked your original shortened / redirected URL, and now you’ve fixed the robots rules, it looks like the validator is showing a card.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
