Would the following robots.txt block access to /?id=asdf but would still allow / to be indexed?
User-agent: * Disallow: /?id=
ThomasB
5:23 pm on Nov 28, 2005 (gmt 0)
Ratbert, it's unlikely to work.
I'd suggest creating a dedicated filename and exclude it specifically.
alexo
10:10 pm on Nov 30, 2005 (gmt 0)
Hello
pls, correct me, if this is wrong.
i guess, it's that , what u need
User-Agent: * Disallow: /*?
spiral
11:12 am on Dec 1, 2005 (gmt 0)
This is what Google claims, at least for G. bot - see [google.com...]
Receptional Andy
11:18 am on Dec 1, 2005 (gmt 0)
You could add a robots meta tag to the pages you want excluded instead - this would probably be simpler than messing with non-standard robots.txt contents.
If you are only concerned about Google, and want non-standard code, I believe the following might work: