Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Sitemaps cannot find robots.txt

Is it my fault?

         

taps

11:35 am on Apr 3, 2006 (gmt 0)

10+ Year Member



I just wanted to check my robots.txt via Google Sitemaps. But after clicking on "robots.txt" I just received a "not found" message.

This happens for all my sites.

Opening robots.txt via ht*p://mysite.xy/robots.txt works fine.

Has anyone the same problem?

McDowra

8:00 pm on Apr 3, 2006 (gmt 0)

10+ Year Member



I had the same problem with my site earlier, it cleared itself up within a couple days... however google did crawl several pages disallowed by my robots.txt during that time. It doesnt seem that they have indexed any of those files though, with google as turbulent as it seems right now who knows though

BillyS

8:09 pm on Apr 3, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>>Has anyone the same problem?

I'm not sure why, but about a week ago I got a robots.txt "not found" message. I checked the site and it was there - no problem.

This caused quite a few errors in sitemap - all with the robots.txt not found message. They are finally clearing up now. No long term damage (or so it seems).

taps

8:23 pm on Apr 3, 2006 (gmt 0)

10+ Year Member



It seems to be fixed. Google is showing my robots.txt again. Phew :-)