Forum Moderators: goodroi

Message Too Old, No Replies

Disallow dynamic content with mod_rewrite

Disallow dynamic content with mod_rewrite

         

crglmb

6:10 am on Oct 9, 2005 (gmt 0)

10+ Year Member




I have just implemented mod_rewrite on our site.
I want G to index only rewritten content and not
dynamic.
I read on google's site to use this line in robots text to Disallow dynamic php content
User-agent: Google-bot
Disallow: /*?

Does anyone have any experience with this?
Sample of rewritten content
www.mysite.com/17.Products/47.Anchoring

Thank you,
Craig

ThomasB

3:54 pm on Oct 10, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Craig, first off, welcome to WebmasterWorld!

Googles spider is called "Googlebot". So you might change the entry accordingly:

User-Agent: Googlebot
Disallow: /*?