Twitter Validator does not find metatags, but every other web scraper does

[rborn]

Description of issue:
Twitter does not show a card for the specified URL even though all of the OG and Twitter metatags are present.

The card validator states:
INFO: Page fetched successfully
WARN: No metatags found

URL affected (must be public):

The Future of Concert Sponsorship for Music Venues

view-source:https://shha.re/thXw5

Troubleshooting steps attempted:

1. Twitter Card metatag is present

2. Getting the webpage as Twitterbot (using curl) still shows metatags

3. Metatags are static – there is no manipulation client side

4. The robots.txt file allows Twitterbot
   curl http://shha.re/robots.txt

User-agent: Twitterbot
Disallow:
Allow: *

The image hoster/domain also allows Twitterbot

curl https://pcore-customer-media.s3.amazonaws.com/robots.txt

User-agent: Twitterbot
Disallow:
Allow: *

5. Yes, we are using a current, valid twitter:card – ‘summary_large_image’

6. Image URL is public and absolute

The image falls within necessary dimension requirements.
2092x1396
388KB

7. SSL is valid and page was fetched successfully.

8. Yes, the URL is a fully qualified DNS.

Thank you in advance for your help!

[andypiper]

I am currently hitting an instant SSL error when attempting to curl that URL.

[rborn]

Hey @andypiper,

Thanks for the quick response!

What part of the certificate is failing for you (CA? Hostname Verifier? etc…)? For me, the whole chain checks out.

• Connected to shha.re (107.23.14.142) port 443 (#0)
• TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
• Server certificate: shha.re
• Server certificate: Amazon
• Server certificate: Amazon Root CA 1
• Server certificate: Starfield Services Root Certificate Authority - G2

[andypiper]

I’m running iCurlHTTP on an iPad and if I disable the SSLv3 check it works. if that is enabled, it fails. I am not in the office right now so I cannot comment on whta the craweler might be having issues with.

[rborn]

If it was an SSL issue, would the crawler/validator still be able to fetch the page?

The card validator states:
INFO: Page fetched successfully
WARN: No metatags found

In terms of SSL and architecture, it’s the same configuration we’re using for other services (e.g. cloudcampaign.io) that populate Twitter cards fine.

[andypiper]

There is no Content-Type header on the server response either, that would be a high likelihood of why the tags are not being pulled in.

[rborn]

Ah, that could be it. Thanks so much, I’ll add that now!

Is this the correct content-type? “text/html; charset=UTF-8”

[andypiper]

that would work I think.

[rborn]

@andypiper – you are a saint! Honestly, thank you so much.

I have been pulling my hair out over this all afternoon. It works now.

Thanks again!

[system]

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.