homepage Welcome to WebmasterWorld Guest from 54.167.179.48
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
googlebot not get robots.txt
robots.txt
Luka2013



 
Msg#: 4535792 posted 1:41 am on Jan 14, 2013 (gmt 0)

Hio every one.

In Google Webmaster I display this message
Network unreachable: robots.txt unreachableWe were unable to crawl your Sitemap because we found a robots.txt file at the root of your site but were unable to download it. Please ensure that it is accessible or remove it completely.And the Fetch as Googlebot feature complains it cannot get robots.txt

But robots,txt is ok
but for others search engine site map work very good.

[edited by: incrediBILL at 3:38 am (utc) on Jan 14, 2013]
[edit reason] removed URLs, no specifics please [/edit]

 

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4535792 posted 3:43 am on Jan 14, 2013 (gmt 0)

But robots,txt is ok

Was that a typo?

This is a more serious question than English speakers may realize. That period on your numeric keypad isn't a physical period, it's a "decimal separator" which may be silently replaced by a physical comma if you've got a non-English operating system. (By weird coincidence I have just this instant come from posting about games. It was a game-registration-code problem that first led me to the decimal-separator explanation.)

stuattpoul



 
Msg#: 4535792 posted 11:18 am on Feb 5, 2013 (gmt 0)

read this article for more help...
[support.google.com...]

khaty



 
Msg#: 4535792 posted 7:52 am on Feb 28, 2013 (gmt 0)

Is it really the robots.txt or maybe you placed meta="no index, nofollow" in your website. This happened to my website before they spiders unable to crawl my website too.

phranque

WebmasterWorld Administrator phranque us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4535792 posted 1:06 am on Mar 2, 2013 (gmt 0)

welcome to WebmasterWorld, Luka2013!

check your server log files for clues about what happened to the googlebot request.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved