homepage Welcome to WebmasterWorld Guest from 54.227.56.174
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Code, Content, and Presentation / Apache Web Server
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL & phranque

Apache Web Server Forum

    
Google found actual scripts even though I have URL rewrite!
apauto




msg:4232798
 7:33 pm on Nov 20, 2010 (gmt 0)

We rewrite all of our URLs using mod_rewrite with htaccess.

Somehow google found the actual pages that the rewrites use, and so now there is duplicate content on every page.

Our rewrites are done in a simple way:

RewriteRule ^page/(.*).html$ /scripts/widget.php?color=$1 [L]

We checked and checked and there is no link anywhere that does not use the rewritten URLs.

Does anyone know any way this might have happened, so we can fix his? Apparently if you type in the rewritten URL into the browser, on some rare occassions it redirects you to the script URL. I've only been able to replicate this one. No idea how this happens, but this must have been how Google discovered this.

Any help is appreciated, thanks!

 

g1smd




msg:4232849
 10:15 pm on Nov 20, 2010 (gmt 0)

Yes, you can set a redirect to fix this. It's one of the most popular topics in this forum. There's hundreds of prior threads with example code.

Additionally your existing code would benefit with the (.*) pattern changing to ([^.]+) or similar. It will operate a lot more quickly.

apauto




msg:4232898
 1:09 am on Nov 21, 2010 (gmt 0)

Thanks g1smd! Can you give me a search term to search for? I keep getting irrelevant results... thanks!

apauto




msg:4233631
 4:14 pm on Nov 22, 2010 (gmt 0)

Anyone know a search term? :)

apauto




msg:4234713
 11:26 pm on Nov 24, 2010 (gmt 0)

c'mon guys, help out a newbie :)

cmnetworx




msg:4235570
 7:47 am on Nov 27, 2010 (gmt 0)

Some good examples and detailed info on htaccess usages

[corz.org ]

jdMorgan




msg:4237619
 10:34 pm on Dec 1, 2010 (gmt 0)

Please see our Apache Forum Charter and our Apache Forum Library for (a lot) more info.

# Externally redirect only direct client requests for script filepath back to search-friendly URL
RewriteCond %{THE_REQUEST} ^[A-Z]+\ /scripts/widget\.php\?color=([^&\ ]+)\ HTTP/
RewriteRule ^scripts/widget\.php$ http://www.example.com/page/%1.html? [R=301,L]
#
# Internally rewrite search-friendly widget requests to script
RewriteRule ^page/([^.]+)\.html$ /scripts/widget.php?color=$1 [L]

This code works if and only if "color" is the only query string parameter. For additional parameter support, you will need to provide more information.

Jim

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Code, Content, and Presentation / Apache Web Server
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved