Forum Moderators: goodroi

Message Too Old, No Replies

Syntax check robots.txt

There should be atleast 1 disallow line in any Robots.txt.

         

mycutegoddess

3:32 pm on Mar 16, 2006 (gmt 0)

10+ Year Member



1 ERROR There should be atleast 1 disallow line in any Robots.txt.

User-agent: *Disallow: /somefolder/somefile.aspx

We're sorry, this robots.txt does NOT validate.
Warnings Detected: 1
Errors Detected: 1

1 warning An empty user agent field was detected. Each User-Agent record should have atleast one disallow line per record. This error may have also been generated due to bad line enders.

------------------------------------

User-agent: *Disallow: /somefolder/somefile.aspx

------------------------------------
What I really want to know is if robots are seeing this robot.txt line will it react to such a error code...

Have anyone experienced to this situation? Pls give me an explaination...

Thanks...

Dijkgraaf

6:09 am on Mar 17, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



By the fact that it seems to be running into on line
User-agent: *Disallow: /somefolder/somefile.aspx
rather than

User-agent: *
Disallow: /somefolder/somefile.aspx

I suspect that you are missing a line break or a carriage return.
What editor did you use to create the file?

mycutegoddess

10:32 am on Mar 17, 2006 (gmt 0)

10+ Year Member



I edited it by hand.. but forgot to separate in two line...

I've check this by searchengineworld dot com and it return error message. So what I'm wondering is robot.txt feature in google sitemap have shown no error.

What I really want to know is it will be affect to robot when they see the error and leave my site immediately...

Thanks,