homepage Welcome to WebmasterWorld Guest from 54.167.182.201
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

This 35 message thread spans 2 pages: < < 35 ( 1 [2]     
"Googlebot can't access your site"
IanCP

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4497999 posted 1:02 am on Sep 21, 2012 (gmt 0)

Ted told me I should be able to search for this topic discussed here recently. Unfortunately I can find nothing at all.

I received an email from Google with the subject "Googlebot can't access your site http://example.com/".

Which of course is rubbish. I rarely look at webmaster Tools but for quite some time now Google has had my site under two names.

http://www.example.com/ [correct] and;

http://example.com/ [incorrect] and the thrust of the email from them.

Over the last 24 hours, Googlebot encountered 1 errors while attempting to connect to your site http://example.com/. Your site's overall connection failure rate is 50.0%. You can see more details about these errors in Webmaster Tools


Remedy anyone?

Thanks

 

phranque

WebmasterWorld Administrator phranque us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4497999 posted 8:05 am on Sep 24, 2012 (gmt 0)

The server would have to perform some equivalent of the -d and -f test on every request, and pore over your htaccess to make sure the request isn't coming from someone who will end up being blocked (core comes after all mods including rewrite). You're looking at a significant detour into a php script for every single request, because a server-level redirect on its own would happen before the request ever reaches your individual site.

someone has to do the work eventually.
the best solution is to host both virtual hostnames on the same server so you can do this work for the first request instead of delaying it until the subsequent request.

triggerfinger

5+ Year Member



 
Msg#: 4497999 posted 5:32 pm on Sep 24, 2012 (gmt 0)

I just wanted to chime in here and say that I have been seeing the same things. We get constant errors in GWT like the ones described above. Even weirder, we are getting requests for strange robots.txts.

vanityURL -> 301 -> deepurl.html
logfiles request -> deepurl.html/robots.txt

Completely ridiculous, I'm pretty sure this is an issue with GWT if not google's crawler.

IanCP

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4497999 posted 8:42 pm on Sep 24, 2012 (gmt 0)

We get constant errors in GWT like the ones described above

For another site, for robots.txt I often get this from GWT, Another "Googlebot can't access your site":

http://www.example2.com//robots.txt

Note the double forward slash, I just do a manual fetch and the GWT becomes a happy face.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4497999 posted 10:44 pm on Sep 24, 2012 (gmt 0)

the best solution is to host both virtual hostnames on the same server

Well, I was talking specifically about shared hosting. Different set of choices. Can't remember what OP's situation was, since the lead-off post doesn't say one way or the other.

Andy Langton

WebmasterWorld Senior Member andy_langton us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4497999 posted 11:22 pm on Sep 24, 2012 (gmt 0)

We get constant errors in GWT like the ones described above


I'm certainly no stranger to crazy crawl behaviour from Google, but I'd be a bit wary that this isn't as a result of an unintended response from your own server (e.g. a redirect).

The problem is that Google doesn't provide references for most data in GWT (or complete data!) and analysing it yourself can be a difficult exercise without the right tools.

This 35 message thread spans 2 pages: < < 35 ( 1 [2]
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved