Welcome to WebmasterWorld Guest from

Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

mod_rewrite and Google

using mod_rewrite with safety

3:42 am on Jun 23, 2005 (gmt 0)

New User

10+ Year Member

joined:July 3, 2004
votes: 0

I've made some SEO changes in my website and now, I'm using mod_rewrite. Do I've to remove all dynamic indexed pages from Google?

if yes, is this the right code for robots.txt file :

User-agent: Googlebot
Disallow: /*?

thanx in advance.

9:55 am on June 23, 2005 (gmt 0)

Full Member

joined:Jan 12, 2004
votes: 0

User-agent: Googlebot
Disallow: /FoldernameOrPagename/WhateverIfAny


10:31 am on June 23, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Oct 30, 2004
votes: 0

jorje: I think that should work. This is indeed what Google recommends to remove dynamic content.

However: I'd be careful. You'll have more control if you name the files explicitly (sp?). This might be a little more work but a lot more control:

User-agent: Googlebot
Disallow: /show.php
Disallow: /otherdynamicthing.php

10:50 am on June 23, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 9, 2003
votes: 0

I did this and it did very poorly.

I disallowed all dynamic content for several VBulletins after I did a rewrite. Google had a hard time ranking anything. When I let them back in, they were able to do a much better job of ranking the pages.

4:38 am on June 24, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:June 24, 2005
votes: 0

my site uses?src=___ on many urls and google and other search engines are seeing these as different pages with the same content which can lead to disaster. is it possible to do a mod_rewrite with wildcards on the src marker?