homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

mod_rewrite and Google
using mod_rewrite with safety

 3:42 am on Jun 23, 2005 (gmt 0)

I've made some SEO changes in my website and now, I'm using mod_rewrite. Do I've to remove all dynamic indexed pages from Google?

if yes, is this the right code for robots.txt file :

User-agent: Googlebot
Disallow: /*?

thanx in advance.



 9:55 am on Jun 23, 2005 (gmt 0)

User-agent: Googlebot
Disallow: /FoldernameOrPagename/WhateverIfAny



 10:31 am on Jun 23, 2005 (gmt 0)

jorje: I think that should work. This is indeed what Google recommends to remove dynamic content.

However: I'd be careful. You'll have more control if you name the files explicitly (sp?). This might be a little more work but a lot more control:

User-agent: Googlebot
Disallow: /show.php
Disallow: /otherdynamicthing.php


 10:50 am on Jun 23, 2005 (gmt 0)

I did this and it did very poorly.

I disallowed all dynamic content for several VBulletins after I did a rewrite. Google had a hard time ranking anything. When I let them back in, they were able to do a much better job of ranking the pages.


 4:38 am on Jun 24, 2005 (gmt 0)

my site uses?src=___ on many urls and google and other search engines are seeing these as different pages with the same content which can lead to disaster. is it possible to do a mod_rewrite with wildcards on the src marker?


Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved