1script
msg:4498036 | 2:37 am on Sep 21, 2012 (gmt 0) |
No remedy but perhaps it will make you feel a little better: you are not alone! I'm getting the same notifications on a number of sites in my account and what they say is certainly not true: not only DNS was up all the time but also, why other sites, also served by the same DNS server, are not affected?
The funny part is: if 1 error is 50% of all attempts, then they only tried to load 2 pages from that site on that day? That's a bit too low for Googlebot, no matter how small and even how badly penalized the site it.
|
g1smd
msg:4498038 | 2:42 am on Sep 21, 2012 (gmt 0) |
You should register both www and non-www in WMT. Google gives separate crawl reports and crawl errors for each one.
It may have been a temporary glitch with the site, or may have be a bottleneck nearer to Google's end.
|
IanCP
msg:4498061 | 3:50 am on Sep 21, 2012 (gmt 0) |
| You should register both www and non-www in WMT |
|
The proper one has been there quite a long time. The non-www one Google did that all by themselves quite some time ago, hence the problem. I didn't put it into WMT. Strange.
Simply put: http://example.com/ is non-existent, so they can't crawl what doesn't exist.
|
IanCP
msg:4498062 | 3:51 am on Sep 21, 2012 (gmt 0) |
I'll await more answers but my gut feeling is simply to delete it.
|
tedster
msg:4498071 | 4:19 am on Sep 21, 2012 (gmt 0) |
What response does the server give when a regular browser asks for http://example.com/
|
Sally Stitts
msg:4498082 | 5:40 am on Sep 21, 2012 (gmt 0) |
Similar thing here.
In Webmaster Tools, under Health - Crawl Errors - Site Errors
All day long, I get -
DNS - Couldn't communicate with the DNS server
Server Connectivity - Request timed out or site is blocking Google, and
Robots.txt Fetch - Crawl postponed because Robots.txt was inaccessible.
I contacted my host, and they said there is no issue on our end.
All regular pages are working just fine.
Is Google messing up here?
.
|
tedster
msg:4498089 | 6:09 am on Sep 21, 2012 (gmt 0) |
Thw only thing I can think of is that somewhere on the web there's a link to example.com without the "www". Whatever the cause, it's certainly not going to cause a ranking problem if the hostname doesn't exist. You don't want it to rank anyway!
|
indyank
msg:4498090 | 6:11 am on Sep 21, 2012 (gmt 0) |
Yes I am hearing this from several people and also seeing this on a number of sites. Definitely looks like something wrong at their end as logs and monitoring services don't report any downtime.
|
lucy24
msg:4498095 | 6:25 am on Sep 21, 2012 (gmt 0) |
| my gut feeling is simply to delete it |
|
You mean delete the non-canonical domain from wmt? No. It will just come back. There's a place in gwt where you tell them your preferred name. Once they've assimilated it, future SERPs will eventually give only that form.
You also need to make sure there's a redirect in place from all wrong forms of your domain name to the one right form. That's in htaccess or config, not in gwt. There are, at a rough guess, eight thousand posts in the Apache forum telling how to do it. Or, if you're on shared hosting, you can cheat and let your host do it for you. They should have a checkbox somewhere.
|
tedster
msg:4498097 | 6:30 am on Sep 21, 2012 (gmt 0) |
Links to the "naked" domain name are relatively common on the web, I'd think. Registrars listing Whois info for domains they registered, for example. I think this one might haunt you until you take Lucy's advice.
|
IanCP
msg:4498121 | 7:49 am on Sep 21, 2012 (gmt 0) |
@tedster
| What response does the server give when a regular browser asks for http://example.com/ |
|
Checking right now in FF.
| Oops! Firefox could not find .example.com |
|
From memory, ditto in IE.
@lucy24
| You mean delete the non-canonical domain from wmt? No. It will just come back. |
|
The very reason I said I'll await further replies for guidance.
|
lucy24
msg:4498188 | 10:05 am on Sep 21, 2012 (gmt 0) |
| Oops! Firefox could not find .example.com |
|
Oops indeed. Did you really type the url with a leading period, or did it just sneak into the post? Quick detour confirms that a leading . makes the difference between a variant name (example.com) and complete failure (.example.com doesn't exist).
Except that if I just type . and nothing more, the browser will helpfully supply the whole URL of this current thread.
|
indyank
msg:4498191 | 10:31 am on Sep 21, 2012 (gmt 0) |
The non-www one Google did that all by themselves quite some time ago, hence the problem. I didn't put it into WMT. Strange.
Simply put: http://example.com/ is non-existent, so they can't crawl what doesn't exist. |
|
AFAIK that isn't the real issue. I have 301 redirected all other url forms to the correct ones and yet i see those google notifications for both the domain versions. Similar notifications were received by several other people I know.
|
bsand715
msg:4498195 | 10:43 am on Sep 21, 2012 (gmt 0) |
Yesterday I received one for a site I do not own or control. But several years ago, I did down load it on my cumputer to do some work on it for a friend. It is still in my C files, only place I can find that would assosiate the site to myself.
|
morganstanly
msg:4498200 | 10:50 am on Sep 21, 2012 (gmt 0) |
I think you have not used 301 redirect for your website. Used 301 Redirect and see your result anytime in any search engine see only one results with (www.). And add sitemap.xml and verify in webmaster tools.
|
1script
msg:4498295 | 4:01 pm on Sep 21, 2012 (gmt 0) |
By the way: these errors don't only come to the non-www versions of domains. I've received a few for the proper www version. This probably means that that www domain is doing so poorly on Google that 1 visit is really-really 50% of the entire daily crawl budget. At least in my case it looks that way. So, I can actually believe that 1 visit could be 50% of the daily crawl budget for non-www domain if Google knows that this is NOT the canonical version.
Why do you need to crawl it then is beyond me. Perhaps for some technical reason that has nothing to do with ranking or indexing, perhaps some registrar-related business.
This made me think that the DNS error I've received on the proper www version probably means that for this domain Google has mixed up which is canonical and they probably think www is actually not the canonical version. Gotta go take a look!
|
IanCP
msg:4498457 | 9:51 pm on Sep 21, 2012 (gmt 0) |
@lucy24
| Did you really type the url with a leading period |
|
Nope. I literally put in http://example.com/
@morganstanly
| Used 301 Redirect and see your result anytime in any search engine see only one results with (www.). And add sitemap.xml and verify in webmaster tools. |
|
That's an impossibility to do with http://example.com/ simply because it doesn't exist.
My take on this is that somewhere out there someone has place a link to http://example.com/blue-widegets.htm and that is the cause.
Similarly I've often had problems with Google and one or more dodgy links I tracked down to a blog somewhere:
http://www.example.com/green-widegets.htm[full stop or question mark]
Oh well.
|
Andy Langton
msg:4498480 | 11:36 pm on Sep 21, 2012 (gmt 0) |
| That's an impossibility to do with http://example.com/ simply because it doesn't exist. |
|
By "doesn't exist" I assume you mean that there is no DNS entry. But it does exist because it's registered and you are using subdomains (www) of it.
Basically, the web (and Google) expect websites to work without a www. The message is actually a hint from Google that there is some value in creating the necessary DNS entry and redirecting without the www to the www - value enough that Google is crawling it, anyway. the simple fix is to create the redirect and then forget about it. if you don't want to do so, then as long as Google perceive value, you'll keep getting the error.
|
IanCP
msg:4498501 | 1:59 am on Sep 22, 2012 (gmt 0) |
Thanks Andy, I'll flick pass it to my support people to fathom it out as there is nothing I have access to in my control panel [as far as I know].
|
lucy24
msg:4498510 | 3:58 am on Sep 22, 2012 (gmt 0) |
| Basically, the web (and Google) expect websites to work without a www. The message is actually a hint from Google that there is some value in creating the necessary DNS entry and redirecting without the www to the www |
|
By exquisite coincidence, my most recent logs illustrate the point:
209.85.238.28 - - [dd/Sep/2012:aa:bb:58 -0700] "GET /google{hexadecimalstuff}.html HTTP/1.1" 301 541 "-" "Google-Site-Verification/1.0"
209.85.238.101 - - [dd/Sep/2012:aa:bb:58 -0700] "GET /google{hexadecimalstuff}.html HTTP/1.1" 200 384 "-" "Google-Site-Verification/1.0"
That's what it wants to see: One redirect, one successful fetch. Both, of course, are the identical file :)
|
phranque
msg:4498992 | 10:00 pm on Sep 23, 2012 (gmt 0) |
| That's an impossibility to do with http://example.com/ simply because it doesn't exist. |
|
at the bare minimum the virtual server at your example.com hostname should "exist enough" to accept a HTTP Request and respond with a sitewide 301 status code redirect to www.example.com which is your canonical hostname.
ideally the server hosting at example.com should be smart enough to redirect only to 200 OK results on www.example.com by avoid redirects to error pages or redirects to additional redirects.
|
Robert Charlton
msg:4499002 | 10:52 pm on Sep 23, 2012 (gmt 0) |
| That's an impossibility to do with http://example.com/ simply because it doesn't exist. |
|
Something doesn't seem to be computing here (no pun intended).
IanCP, I'm going to guess that somewhere you're not understanding some of the technical vocabulary being used here, and that you may not be set up with software to analyze what's going on. Please forgive me if this isn't the case.
Bits of vocabulary can be a steep learning curve, I've found, because many of the people qualified to give you good answers naturally use language that contains terms you're not necessarily familiar with, and I can see where people here might be talking past each other. Let me suggest for now a few things...
- type http://example.com/ (with your domain instead of example.com) into the address bar of your browser and hit enter. What happens? Please report back to us, including the url "address" you see in the address bar.
- do the same with http://www.example.com/ and tell us what happens.
Now, I'm going to mention a specific tool here, something we don't usually do in this forum, but it's already been mentioned in our favorite seo tools thread in this forum....
Favorite SEO Tools
http://www.webmasterworld.com/google/4367011.htm [webmasterworld.com]
The tool is, among other things, a server header checker. It's URI Valet - [urivalet.com...]
Type http://example.com into the URI input field up top and click the Submit button.
Report back the requests and responses listed under "Server Headers Details" further down the page. They'll be in bold blue type.
If your preferred canonical was http://www.example.com, you would get see as the first item"
1. REQUESTING: http://example.com
...and if it properly redirected to http://www.example.com, you would see a blue heading further down the page saying...
SERVER RESPONSE: 301 Moved Permanently
etc... You'd get note that it's redirecting...
Redirecting to http://www.example.com
2. REQUESTING: http://www.example.com
...and then further down the page, you'd get a server response and a destination URI.
Try it also with http://www.example.com in the URI box. You should get at least a somewhat different set of responses.
Please report back on what all those are, and we can go from there.
|
lucy24
msg:4499015 | 11:42 pm on Sep 23, 2012 (gmt 0) |
| ideally the server hosting at example.com should be smart enough to redirect only to 200 OK results |
|
You can do this on your own server, but I really don't think it's viable on shared hosting. The server would have to perform some equivalent of the -d and -f test on every request, and pore over your htaccess to make sure the request isn't coming from someone who will end up being blocked (core comes after all mods including rewrite). You're looking at a significant detour into a php script for every single request, because a server-level redirect on its own would happen before the request ever reaches your individual site.
You can cut back a little bit by doing the 301 yourself instead of letting the host do it. But it still won't affect your core-level lockouts.
But it really isn't that big an issue. I track redirects for other reasons, and the 301-to-403 sequence-- or 301-to-301-to-something-else-- is very rare. In fact the most common occurrence of 301-to-301 is from major search engines that intentionally ask for the wrong form of the name.
|
IanCP
msg:4499022 | 11:55 pm on Sep 23, 2012 (gmt 0) |
@ Robert Charlton:
As advised, I checked http://example.com/
Response:
Server Headers Details
1. REQUESTING: http://example.com/
GET / HTTP/1.1
Accept: */*
Accept-Encoding: gzip
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Accept-Language: en-us,en;q=0.5
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; AIRF; GTB7.4; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET CLR 2.0.50727; .NET CLR 1.1.4322)
Host: example.com
Connection: Keep-Alive
SERVER RESPONSE: 301 Moved Permanently
Date: Sun, 23 Sep 2012 23:44:42 GMT
Server: Apache
Location: http://www.example.com/
Content-Length: 245
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset="iso-8859-1"
X-Pad: avoid browser bug
Redirecting to http://www.example.com/ ...
2. REQUESTING: http://www.example.com/
GET / HTTP/1.1
Accept: */*
Accept-Encoding: gzip
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Accept-Language: en-us,en;q=0.5
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; AIRF; GTB7.4; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET CLR 2.0.50727; .NET CLR 1.1.4322)
Host: www.example.com
Connection: Keep-Alive
SERVER RESPONSE: 200 OK
Date: Sun, 23 Sep 2012 23:44:42 GMT
Server: Apache
Accept-Ranges: bytes
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset="iso-8859-1"
Destination URI: http://www.example.com/ |
|
Then submitting http://www.example.com/ I get this response:
Requested: 2012/09/23 16:51:52
User Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; AIRF; GTB7.4; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET CLR 2.0.50727; .NET CLR 1.1.4322)
Server Headers Details
1. REQUESTING: http://www.example.com/
GET / HTTP/1.1
Accept: */*
Accept-Encoding: gzip
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Accept-Language: en-us,en;q=0.5
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; AIRF; GTB7.4; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET CLR 2.0.50727; .NET CLR 1.1.4322)
Host: www.example.com
Connection: Keep-Alive
SERVER RESPONSE: 200 OK
Date: Sun, 23 Sep 2012 23:51:52 GMT
Server: Apache
Accept-Ranges: bytes
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset="iso-8859-1"
Destination URI: http://www.example.com/ |
|
Thanks very much for the heads-up on the SEO Tools.
|
indyank
msg:4499113 | 4:43 am on Sep 24, 2012 (gmt 0) |
You can see that the first one http://example.com/ is 301 redirected to http://www.example.com/. And that is what you want as your preferred url is http://www.example.com/.
When you requested http://www.example.com/, you are getting a 200 OK response and that is perfect.
So the issue isn't at your end at all.
|
Robert Charlton
msg:4499114 | 4:46 am on Sep 24, 2012 (gmt 0) |
IanCP - Looks like you're redirected just fine. I can't speak to the rest of your canonicalization, but the www subdomain certain appears to be handled properly.
It may have been a glitch, but I wouldn't lose any sleep over it. If you do a search on the email text and see that lots of others are receiving the same note, then it might be a glitch on Google's end, or else a classic bit of unclarity.
Maybe this is their new way of telling you that everything's OK. ;)
|
kemcar
msg:4499118 | 5:14 am on Sep 24, 2012 (gmt 0) |
I believe the www domain was not doing well with Google for the time being for that particular page. So I don't think there is anything you can do to solve it from your end.
|
tedster
msg:4499119 | 5:19 am on Sep 24, 2012 (gmt 0) |
| My take on this is that somewhere out there someone has place a link to http://example.com/blue-widegets.htm |
|
Exactly. For that very reason, you want to make sure that every URL that doesn't include a "www" gets redirected to the exact same file path, but using the "www" hostname. In other words, a redirect only for (or to) the root alone isn't really enough.
So I'd suggest you double check by taking a good internal "with-www" URL and requesting the "no-www" version. It should redirect as well.
|
IanCP
msg:4499142 | 5:54 am on Sep 24, 2012 (gmt 0) |
Thanks to all the good folks for both your generosity of spirit and your time on this issue.
In Australian terms?
"I luv's ya all"
Again thanks, especially for your patience on an issue way beyond my depth.
@ Robert Charlton
I owe you.
@Tedster
Thanks for your guidance.
|
| This 35 message thread spans 2 pages: 35 ( [1] 2 ) > > |
|
|
