homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / WebmasterWorld / Webmaster General
Forum Library, Charter, Moderators: phranque

Webmaster General Forum

"Googlebot can't access your site"
Googlebot encountered 1 errors

 5:16 am on Oct 23, 2012 (gmt 0)

Hello there,
One of my website has encountered an error
http://www.abc.com/: Googlebot can't access your site

The detail message is as
http://www.abc.com/: Googlebot can't access your site
October 20, 2012

Over the last 24 hours, Googlebot encountered 1 errors while attempting to access your robots.txt. To ensure that we didn't crawl any pages listed in that file, we postponed our crawl. Your site's overall robots.txt error rate is 33.3%.

In fact I have uploaded robots.txt file yesterday which was not there before..



 8:18 am on Oct 23, 2012 (gmt 0)

check your server access log file for clues during that period.


 8:23 am on Oct 23, 2012 (gmt 0)

:: counting on fingers ::

Out of three tries, they got it twice and couldn't find it once. Not a problem. They're just waiting for a best three out of three. If they looked repeatedly and could never find a robots.txt, they would assume you really don't have one, and that would have been fine too. It's the mixed answers make the googlebot uneasy.

But not as uneasy as if it met something other than a 404* or 200. Then it just wouldn't dare crawl at all.

* Has anyone in history ever served up a 410 when asked for robots.txt? :)

Global Options:
 top home search open messages active posts  

Home / Forums Index / WebmasterWorld / Webmaster General
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved