Unable to render Card preview

john_020680 · 2017-03-06T13:35:24.976Z

Hi Guys,

I am trying to check Card preview but i am getting this ‘ERROR: Fetching the page failed because it’s denied by robots.txt.’ error.

For this i have added some code in robot.txt:

User-Agent: google
User-Agent: googlebot
User-agent: Twitterbot
Crawl-delay: 10

User-agent: *
Disallow: /

But still getting issue. Can any update help me on this

andypiper · 2017-03-06T14:34:02.519Z

We require a link to the site in order to understand what is happening. Please confirm you’ve been through the troubleshooting post.

john_020680 · 2017-03-06T16:02:34.275Z

Here is link: http://themarketmogul.com/ai-will-not-spell-end-lawyers/

andypiper · 2017-03-06T16:30:29.254Z

$ curl -A Twitterbot http://themarketmogul.com/robots.txt                             
User-Agent: google
User-Agent: googlebot
Disallow: /wp-content/plugins/
Disallow: /readme.html
Disallow: /wp-admin
Crawl-delay: 10

User-agent: *
Disallow: /

You are explicitly preventing Twitterbot from access to your site.

john_020680 · 2017-03-06T16:49:14.425Z

This is older one. Currently setting is : User-Agent: google
User-Agent: googlebot
User-agent: Twitterbot
Disallow: /wp-content/plugins/
Disallow: /readme.html
Disallow: /wp-admin
Crawl-delay: 10

User-agent: *
Disallow: /

john_020680 · 2017-03-06T16:52:31.166Z

Is i need to change any setting on server.

andypiper · 2017-03-06T17:39:12.796Z

Your old robots.txt file must be cached somewhere because I cannot see the version you are referring to.

john_020680 · 2017-03-07T05:07:44.949Z

If you hit this url ‘http://themarketmogul.com/robots.txt’ in browser then you will see the latest one.

john_020680 · 2017-03-07T05:13:17.446Z

And i checked this response using curl hit. I am getting latest robot file. So please check what is the issue now.

andypiper · 2017-03-07T11:34:33.343Z

I’m still seeing the old file - I suspect CloudFlare is caching your older version.

john_020680 · 2017-03-07T12:44:16.780Z

My robot file issue is fixed but in am getting no meta tags found.But in my page source i can see meta tags. here is the url: view-source:http://themarketmogul.com/ai-will-not-spell-end-lawyers/

john_020680 · 2017-03-07T12:52:37.453Z

And i am trying to share this url on twitter http://themarketmogul.com/taming-technology/ but when i check my tweet there is not desciption or image for url

andypiper · 2017-03-07T12:52:56.203Z

You’re providing a link to a 404 page. I’ve just manually validated your other URL and it shows up fine.

john_020680 · 2017-03-07T12:53:34.613Z

http://themarketmogul.com/ai-will-not-spell-end-lawyers/ check this

andypiper · 2017-03-07T12:54:29.875Z

Page not found. I cannot help you. Your site’s cards are working fine, but if you’re trying to post a 404 page then it won’t show a card.

john_020680 · 2017-03-07T12:58:23.229Z

Open this : http://themarketmogul.com/ai-will-not-spell-end-lawyers/

url.png1366x728 59.3 KB

andypiper · 2017-03-07T13:00:15.716Z

Page_not_found_-_The_Market_Mogul.png1952x1434 122 KB

andypiper · 2017-03-07T13:01:13.747Z

I’m unable to help further. Cards on your other pages render just fine when the URLs are shared. Good luck in figuring out what is going on there (your logs may be helpful)

john_020680 · 2017-03-07T13:01:33.378Z

And i am trying to tweet a url and did not get image and description

Twitter.png1366x728 121 KB

there

andypiper · 2017-03-07T15:43:25.472Z

I see cards all the way down your account timeline at the moment. Are you using any ad blockers or other scripting tools that might interfere with Twitter’s web Javascript?