Welcome to WebmasterWorld Guest from

Message Too Old, No Replies

mod_rewrite and Google

using mod_rewrite with safety



3:42 am on Jun 23, 2005 (gmt 0)

10+ Year Member

I've made some SEO changes in my website and now, I'm using mod_rewrite. Do I've to remove all dynamic indexed pages from Google?

if yes, is this the right code for robots.txt file :

User-agent: Googlebot
Disallow: /*?

thanx in advance.


9:55 am on Jun 23, 2005 (gmt 0)

User-agent: Googlebot
Disallow: /FoldernameOrPagename/WhateverIfAny



10:31 am on Jun 23, 2005 (gmt 0)

10+ Year Member

jorje: I think that should work. This is indeed what Google recommends to remove dynamic content.

However: I'd be careful. You'll have more control if you name the files explicitly (sp?). This might be a little more work but a lot more control:

User-agent: Googlebot
Disallow: /show.php
Disallow: /otherdynamicthing.php


10:50 am on Jun 23, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

I did this and it did very poorly.

I disallowed all dynamic content for several VBulletins after I did a rewrite. Google had a hard time ranking anything. When I let them back in, they were able to do a much better job of ranking the pages.


4:38 am on Jun 24, 2005 (gmt 0)

10+ Year Member

my site uses?src=___ on many urls and google and other search engines are seeing these as different pages with the same content which can lead to disaster. is it possible to do a mod_rewrite with wildcards on the src marker?



Featured Threads

Hot Threads This Week

Hot Threads This Month