Card validator suddenly refusing to show previews

[knutmt]

Just recently, Twitter stopped showing our Twitter cards for ALL of our pages. We get the error message: “ERROR: Fetching the page failed because the response is too large.”. The content on the front page is quite small.

Could this be due to some other error than just the size of the content?

Our URL is: https://restdb.io

Any ideas?

[andypiper]

The HTTP headers are over 8192 bytes, which is causing the crawler some issues.

[knutmt]

Around 3kb as far as I can see. How did you measure?

[andypiper]

I used curl -v -A Twitterbot to pull the page, then copied the HTTP headers on the response into this tool, which tells me they are around 8357 bytes (starting at x-xss-protection and ending at date). The three long csp blocks take up most of that, by the looks of it.

[knutmt]

Looks like you’re right. I find it strange because this worked a few days ago.

Any way to fix this except lowering the security of our site?

[knutmt]

Looks like the security headers are repeated three times. I’ll dig deeper. Thanks for the help!

[andypiper]

Looks like x-content-security-policy and x-webkit-csp are deprecated.

[knutmt]

Hi, I resolved this issue by changing the settings on a NodeJS library we use (helmet).

Thanks so much for the help

[andypiper]