robots.txt syntax

'$' and '?'

[sssweb]

I was recently instructed to add the following lines to my robots.txt file:

Disallow: /page.htm$
Disallow: /index.php?

Regarding the '$' ending the first line, is that a valid symbol, and if so, what does it mean. I can't find any resources that mention it, yet it passed a validation test.

Regarding the '?' ending line 2, will that allow bots to spider /index.php while telling them to avoid say, /index.php?var=123

[goodroi]

hiya sssweb,

it sounds like you are referring to adding wildcards to robots.txt. the official robots.txt protocol does not support it. Google and Yahoo have both gone on record as supporting wildcards (aka pattern matching). the smaller bots generally do not support it.

More Google Info
[google.com...]

More Yahoo Info
[ysearchblog.com...]

Official Protocol
[robotstxt.org...]