Why GSC Shows Crawl Errors When Page Loads Normally

  • Share

Google’s John Mueller answered a query about Google Search Console (GSC) displaying crawl errors on pages that load wonderful within the browser. John Mueller defined that the issue is often discovered on the server aspect and never a problem with Googlebot.

The individual asking the query tried validating the online pages in Google Search Console (GSC) however the pages would stubbornly not validate. It’s like one thing was blocking Googlebot and serving an error message.

Here is the query:

“I had a server error on some of my pages. When I checked the few pages shown in the example they worked just fine.

I also used the validate option a few times but Google keeps marking the pages with an error.

It’s been a month since then.

I’ve waited for Googlebot to index these pages with no success.

This has affected my organic impressions and clicks.

Is there anything I can do here?”

John Mueller affirmed that if GSC reveals server errors when Googlebot is crawling the online pages then they actually exist and will not be a bug on Google’s aspect.

Advertisement

Continue Reading Below

Here’s what John mentioned:

“We don’t invent errors on pages. When Googlebot checks the page and there’s a server error then we really see a server error there.”

Temporary Crawl Errors

Next Mueller noticed that a few of these issues are short-term.

Here’s what he mentioned:

“And it might be that this is something that is temporary on your website and if it’s temporary then with one of the future crawls we will …try that again.

And if the error is gone then we can index the page normally.”

Sometimes these errors are certainly short-term.

For instance, a server may go down for upkeep, there may be points in DNS that’s taking down part of the Internet or it may also be that the server is overloaded and stopping crawling.

But that’s not the problem that the individual asking the query is experiencing.

If the online pages load usually however everytime you validate with GSC (or attempt to take a look at it with one in all Google’s instruments just like the wealthy outcomes tester or cellular pleasant tester) and it fails to validate, that’s a problem on the server.

Advertisement

Continue Reading Below

Server Issues Can Cause GSC Indexing Errors

John Mueller subsequent instructed that the issue the individual asking the query is experiencing could also be server associated.

Mueller:

“So that’s something where if you see these kinds of issues come up regularly and in particular you use the validate feature in search console and the validation comes back and says there are still server errors then that’s something I would take up with your hoster.

And try to see if there’s something that they can do to diagnose this issue to double check what might be happening here to give you a sense of how many URLs does this affect, actually.

Because sometimes it can be tricky where if Googlebot is crawling millions of pages from your website and one hundred of them have a server error then probably that’s irrelevant because like there are always some errors somewhere.

But if Googlebot is crawling 200 pages from your website and a hundred of those are server errors then that’s a little bit more concerning that something you’d probably want to fix and make sure that it doesn’t happen.

So that’s kind of the direction I would head there.

It’s not something that Google can fix.

It’s really something that you need to fix on your side with regards to the hosting.”

Diagnosing Googlebot Crawl Errors

There’s a diagnostic trick that you are able to do to establish if this can be a server-wide configuration challenge, in case your IP is being shared by different websites on the server.

What might be finished is to establish the IP tackle that the positioning is on then run that ISP by means of a reverse IP checker that may present what different websites are hosted on that IP tackle.

Then you’re taking that listing of domains and run them by means of one in all Google’s instruments just like the AMP checker or Google’s wealthy outcomes checker instrument.

If the instrument reviews an error response for a number of of the domains then which will point out that there’s a server-wide error, which

Every server has a server log and that’s place to begin diagnosing what’s inflicting the issue. Those server logs will present the date and actual time of when an error occurred in addition to the IP tackle of the customer that triggered the error.

Advertisement

Continue Reading Below

A typical challenge might be an error in how a firewall is about up which may be set too strict and blocking Google.

If you don’t have entry to the server logs then a name to the webhosting buyer help is so as, as John Mueller really useful.

Citation

Watch the Google Office Hours Hangout

Why Does GSC Show Server Errors on Pages that Load Fine?
Watch on the 47:05 minute mark

if( !ss_u ){

!function(f,b,e,v,n,t,s)
{if(f.fbq)return;n=f.fbq=function(){n.callMethod?
n.callMethod.apply(n,arguments):n.queue.push(arguments)};
if(!f._fbq)f._fbq=n;n.push=n;n.loaded=!0;n.version=’2.0′;
n.queue=[];t=b.createElement(e);t.async=!0;
t.src=v;s=b.getElementsByTagName(e)[0];
s.parentNode.insertBefore(t,s)}(window,document,’script’,
‘https://connect.facebook.net/en_US/fbevents.js’);

if( typeof sopp !== “undefined” && sopp === ‘yes’ ){
fbq(‘dataProcessingOptions’, [‘LDU’], 1, 1000);
}else{
fbq(‘dataProcessingOptions’, []);
}

fbq(‘init’, ‘1321385257908563’);

fbq(‘track’, ‘PageView’);

fbq(‘trackSingle’, ‘1321385257908563’, ‘ViewContent’, {
content_name: ‘why-gsc-shows-crawl-errors-when-page-loads-normally’,
content_category: ‘news seo ‘
});

}// end of scroll user

  • Share