Cannot find error

Robots.txt syntax error

[webapache]

I am getting errors in my robots.txt file.

Please help me to correct these errors if some one can .... I created this by using a tool.

User-Agent: *
Disallow: /cgi-bin/
Disallow: /pic/
Disallow: /images/
Disallow: /netforward/
Disallow: /reciprocal/
Disallow: /autorank/
Disallow: /members/

[edited by: oilman at 11:55 pm (utc) on Mar. 13, 2004]
[edit reason] no urls please [/edit]

[closed]

Doesn't look like there's anything wrong. What did you use to check it for errors, and what errors did you get?

[kevinpate]

First, leave off the self promoting url's (a mod will strip it out if you don't do it first)

Second, what you've posted doesn't match the actual robots file.

Try User-agent*
Instead of User-Agent: *

Delete this line:
Disallow:User-agent: *

I may be wrong, but I think you need to dump these three lines also:
Disallow: *.gif
Disallow: *.jpg
Disallow: *.bmp

[webapache]

Thanks for your help ...... the error was not with self promoting urls. As you said agent should be spelled with all lower case and second the / at the end of each directory is not allowed as it is considered as second entry ..... Now I have no errors .....

Question now rises .... is what is Mozilla? is this a robot or what?

[closed]

What kevinpate means by the self promoting urls is that you're not supposed to post links to sites that you own because it's a violation of this site's TOS.

The capital "A" in User-Agent was definitely a problem. I guess I missed that.

AFAIK, the use of wildcards in Disallows are only allowed as extensions to the robots.txt standard. Googlebot, for example, has an extension that makes Disallows like this legal:

Disallow: /*.gif$

webapache:

the / at the end of each directory is not allowed as it is considered as second entry

Putting the / at the end of each directory is allowed and legal.

Mozilla is usually a human visitor, but there are also automated visitors that use it in their UA string.