GSC >Crawl Errors >Server Connectivity

[keyplyr]

For the last 3 days, GSC reports server connectivity errors on one of my sites on a shared server. Googlebot has vanished from the access logs.

However, the Google Page Speed tool has no issue, nor the Mobile-Friendly test tool. Seems to be just Googlebot, Fetch as Googlebot and the robots.txt tester which uses Googlebot.

Now, the site is up and running just fine. Fetch as Bingbot returns a 200 OK, as do browsers. Waiting for server admin to take a look.

Posting here in case anyone else is seeing a similar anomaly. Also posted in Google Groups Webmaster Central Help Forum.

[justpassing]

shared server

May be the host was blocking googlebot somehow. Or if the host has automatic system to blacklist IPs / UA / etc... may be it was caught "accidentally".

I do log everything , and keep reviewing what is blocked, in case of a false positive, or bug in my filtering.

That being said, the last 3 days, Googlebot activity halved at my bigger site. At this site, Googlebot visits 1000 pages per hour all day / week / year long, very constant (+/- 10%). Then 3 days ago, it cut to 500 pages/hour "only", and stayed there for the last 3 days. Now, since 6 hours, it's back to 1000 pages. But it might be coincidence, and no error in the GSC.

[keyplyr]

Google Support volunteers replied that the Fetch as Googlebot tool has "artifacts." I guess that's their way of saying it has Bugs and doesn't always work properly. Well we knew that :)

But the issue is the Crawl Errors > Server Connectivity report

I am now seeing Googlebot in server logs again, so not seeing it for a couple days was likely a coincidence.



[fix typo]

[edited by: keyplyr at 3:16 am (utc) on Oct 24, 2018]

[justpassing]

artifacts

neat, I'll remember it.

so not seeing it for a couple days

is scaring :)

[keyplyr]

Well what scared me is after looking through my server access log a few days ago and not seeing a valid Googlebot hit anywhere, I immediately went to GSC Dashboard where I saw the Crawl Error Report and for the first time ever, in red letters "Server Connectivity Errors" with 312 under it.

Whoa! What happened here? So I started looking through error logs at my server, doing numerous access tests, filing support tickets with my Server Admin, going to Google Support Forum searching through topics and eventually opening a thread, going back to GSC and using other tools, etc.

All coming down to the Crawl Error Report giving false negatives.