Fetching the url failed due to internal error

Retweetedi · 2013-11-18T22:07:26.000Z

I’ve been trying to validate cards for days now.
Does anyone get this error from the validator, “Fetching the url failed due to internal error”, and know how to resolve it? Should I not be the only one seeing it it would be something…

rchoi · 2013-11-20T02:33:48.000Z

Hi @Retweetedi,

If you have a test URL, I’m happy to give it a shot and tell you what I see.

Thanks!

Ryan

Retweetedi · 2013-11-20T12:48:09.000Z

Hi thanks. The url would be http://iretweeted.mina-grey.com(/index.php)
The domain itself can’t be fetched by Twitter though. I have to change host, I suspect.
Two domains I had on the same server (shared hosting) had the same problem. After moving them I managed to fetch them.
The url validator has issues 90% of the times (I think for the number of requests), but if one tries to refresh, with some luck, there is a chance to submit.
Would you please help me to confirm that the domain as it is now can’t be fetched? Thanks so much again.

ikomobi · 2013-11-30T18:04:51.000Z

Hey all,

I have precisely the same problem : internal error, no other explanation. Any help would be greatly appreciated. (Domain is in my account’s Bio)

Cheers,
Théo

Retweetedi · 2013-11-30T21:12:37.000Z

I think I can attempt a troubleshooting. Internal error is much better than “fetcher can’t resolve the address”. Just click on refresh (not too much because fetching from the validator is rate limited too) until you get a response from the fetcher. I would also check the robots.txt file. What’s reported on the knowledgebase (allow twitterbot, disallow all) didn’t work for me. I had to allow all. I hope this helps.

rchoi · 2013-11-30T23:54:46.000Z

Thanks @Retweetedi!

jcgaal_ · 2013-12-19T00:26:18.000Z

Hello! I’ve tried to test my url several times and the error I constantly get is: “Fetching the page failed because the response is too large.” I am using the exact same parameters I have used on other sites successfully with no luck with this particular one. Is there any way to know the cause of this issue?

Thanks in advance.

rchoi · 2013-12-19T07:24:00.000Z

We only can retrieve pages < 2MB or so. Look to reduce page size (and image sizes), if possible.

Retweetedi · 2013-12-19T07:42:29.000Z

I had no idea that web pages could be bigger than 2MB… Thanks @rchoi will check mine as well

Retweetedi · 2013-12-19T13:04:37.000Z

pas de quoi, mais le café doit être italien .Le café français a un goût étrange. Quand vous êtes prêt, je suis disponible (eheheh)

jcgaal_ · 2013-12-27T20:55:02.000Z

Thanks for the response! Could you be a bit more specific as to what do you mean with page size? Isn’t it the purpose of the summary to only grab some information of the entire page?
It’d be great to add that on the documentation page. @rchoi

Retweetedi · 2013-12-28T00:43:10.000Z

well I think @rchoi means that the fetcher grabs the whole page (with cURL or anything like that). If the page is too big, it won’t be able to fetch it correctly. Just an educated guess. It would be useful to add to the documentation this and other types of errors I have seen and aren’t documented. I am not sure it’s possible - for security reasons(?)

logansease · 2014-01-04T04:45:00.000Z

I’m getting this error trying to validate my test url. I’ve validated with the html5 validator… not sure why I’m getting the internal error. Any ideas?

HealthyUKStore · 2014-06-10T12:10:04.000Z

I cant get this to work on www.foreverhealthyliving.co.uk

prascuna · 2014-06-12T13:17:14.000Z

Hi guys, how did you solve that?
I’m getting the same error with this url:

https://www.youniverse.com/api/y/opengraph?id=judge

That’s the full error:
Fetching the page failed due to an internal error, debugId: 590750916942-430439

Cheers

rchoi · 2014-06-16T05:32:51.000Z

Looks like it’s submitted via the validator.

Let us know if you have other questions.

rchoi · 2014-06-16T05:33:47.000Z

I don’t see any twitter:card tag on the site. Are you rendering it only under certain conditions?

prascuna · 2014-06-16T10:35:00.000Z

Hi,
thanks for you reply.
It does print the twitter:card (plus opengraph) and then there’s a javascript that performs a redirect.

You can see that with a:

curl “https://www.youniverse.com/api/y/opengraph?id=judge”

So the idea is basically that twitter scraper will see the html meta tags, and when a user will click on the vcard, he will be redirected (to the home page).

It used to work fine before I switched to https and still works fine accessing from appspot url:

https://0-16-11.youniverse-www.appspot.com/api/y/opengraph?id=judge

So I assume there’s something wrong with the ssl certificate of www.youniverse.com

rvstest6 · 2014-07-03T14:28:13.000Z

Hi,

I’m trying to validate twitter cards, e.g https://app.whipclip.com/twitter4.html, and I get an error from the Twitter Card Validator - “Fetching the page failed due to an internal error, debugId: 4135124398237-381026.”

The page is served via Amazon CloudFront and is stored in Amazon S3.

If I provide the address of the same page directly on S3, there is no error.

Could anyone help identify the problem?

Thanks

jbulava · 2014-07-06T21:53:37.000Z

It looks like an SSL issue. I tried to rule out HTML issues and the W3C validator also has a problem, returning a “500 SSL negotiation failed” error. I would suggest reviewing your DNS records to make sure everything is in order. Unfortunately, I cannot pinpoint why your setup appears to work for all user-agents except for bots.