homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

googlebot not get robots.txt

 1:41 am on Jan 14, 2013 (gmt 0)

Hio every one.

In Google Webmaster I display this message
Network unreachable: robots.txt unreachableWe were unable to crawl your Sitemap because we found a robots.txt file at the root of your site but were unable to download it. Please ensure that it is accessible or remove it completely.And the Fetch as Googlebot feature complains it cannot get robots.txt

But robots,txt is ok
but for others search engine site map work very good.

[edited by: incrediBILL at 3:38 am (utc) on Jan 14, 2013]
[edit reason] removed URLs, no specifics please [/edit]



 3:43 am on Jan 14, 2013 (gmt 0)

But robots,txt is ok

Was that a typo?

This is a more serious question than English speakers may realize. That period on your numeric keypad isn't a physical period, it's a "decimal separator" which may be silently replaced by a physical comma if you've got a non-English operating system. (By weird coincidence I have just this instant come from posting about games. It was a game-registration-code problem that first led me to the decimal-separator explanation.)


 11:18 am on Feb 5, 2013 (gmt 0)

read this article for more help...


 7:52 am on Feb 28, 2013 (gmt 0)

Is it really the robots.txt or maybe you placed meta="no index, nofollow" in your website. This happened to my website before they spiders unable to crawl my website too.


 1:06 am on Mar 2, 2013 (gmt 0)

welcome to WebmasterWorld, Luka2013!

check your server log files for clues about what happened to the googlebot request.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved