KevinC
msg:3783805 | 12:43 am on Nov 11, 2008 (gmt 0) |
I think I found my answer but I'm little nervous pulling the trigger. does this look correct: User-agent: *
Disallow: /admin/
Disallow: /affiliatewiz/
Disallow: /eproducts/
Disallow: /themes/ User-agent: googlebot
User-agent: slurp
User-agent: msnbot
Disallow: /*EMAILFRIEND*
|
goodroi
msg:3784071 | 12:40 pm on Nov 11, 2008 (gmt 0) |
that should work. be aware that wildcards are not official parts of robots.txt protocol. they are special allowances provided by the big search engines. be careful and keep and eye on it just to be safe.
|
KevinC
msg:3788301 | 10:37 pm on Nov 17, 2008 (gmt 0) |
thanks a lot, I have it in place and so far no issues.
|
|
