Forum Moderators: phranque

Message Too Old, No Replies

How to remove duplicate pages?

duplicate pages problem

         

sfseo

9:36 am on Sep 28, 2006 (gmt 0)

10+ Year Member



Yahoo indexed duplicate pages of our homepage! These pages do not exist but yet Yahoo indexed them:

http://www.example.com/?=DSL
http://www.example.com/?=BED

I tried creating a 404 page but was not able to do so using Front Page. The next thing I tried is to use a robots.txt file to Disallow: /?

I need some help. any suggestions would be greatly appreciated. thanks! Chris.

[edited by: jdMorgan at 2:27 pm (utc) on Sep. 28, 2006]
[edit reason] Examplified. NO URLs, please. See TOS. [/edit]

jdMorgan

2:29 pm on Sep 28, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



The solution given here [webmasterworld.com] will accomplish what you want -- with some adjustments. Just redirect the requests to your homepage with the query string removed.

Jim

sfseo

10:36 pm on Sep 28, 2006 (gmt 0)

10+ Year Member



Would this work if I wanted those queries to go to a 404 page? I'm new to this. thx. chris.

RewriteCond %{QUERY_STRING} &?=DSL=[^&]* [NC]
RewriteRule ^?=DSL\.asp$ http://www.example.com/404.asp [R=404,L]

RewriteCond %{QUERY_STRING} &?=BED=[^&]* [NC]
RewriteRule ^?=Bed\.asp$ http://www.example.com/404.asp [R=404,L]

[edited by: jdMorgan at 11:43 pm (utc) on Sep. 28, 2006]
[edit reason] Example.com [/edit]

jdMorgan

11:47 pm on Sep 28, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



No, see the Apache mod_rewrite documentation.

A 404 is not a good solution. Instead, you want to "repair" the requested URL and 301-redirect it to the correct URL. This will both remove the bad URL from search results, and capture the search traffic until the bad URLs are removed from search results.

Jim

sfseo

12:08 am on Sep 29, 2006 (gmt 0)

10+ Year Member



Jd:

Can you please show me an example for http://www.example.com/?=dsl I honestly don't know how to write it and can't figure out how to looking at the examples give from the previous link.

[edited by: jdMorgan at 12:51 am (utc) on Sep. 29, 2006]
[edit reason] Example.com [/edit]

jdMorgan

12:52 am on Sep 29, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Please review our forum charter [webmasterworld.com].

Thanks,
Jim

sfseo

1:08 am on Sep 29, 2006 (gmt 0)

10+ Year Member


Thanks jim. i'm tryin' and learning. This is what i'm trying right now but it's not working...

RewriteCond %{HTTP_HOST}?
RewriteRule ^(.*)$ http:/[smilestopper]/example.com/$1? [R=301,L]

jdMorgan

2:29 am on Sep 29, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



It would be of great help to you to follow those links in the forum charter, and learn about mod_rewrite. The chances of "guessing" code and having it work are essentially zero. And you can cause your site great harm --functionally or ranking-wise-- with one single typo using mod_rewrite or any of several other Apache modules. This code modifies your server configuration and can affect your entire site.

As stated in my first response, the solution in the thread I cited was functionally very close to what you need:

http://www.example.com/?=DSL -- 301 --> http://www.example.com/
http://www.example.com/?=BED -- 301 --> http://www.example.com/


Options +FollowSymLinks
RewriteEngine on
#
RewriteCond %{QUERY_STRING} ^=[A-Z]{3,}
RewriteRule ^$ http://www.example.com/? [R=301,L]

Here, a request for your default document at http://www.example.com/ with an appended query string starting with "=" followed by 3 or more uppercase characters will be permanently-redirected to your default document with the entire query string removed.

This tells Yahoo! (and the other SEs) to drop the bogus URLs they are requesting, and replace them with the correct URL. Any links to the bogus URL will now be credited to the correct URL. Anyone clicking on a bogus URL on Yahoo! or anywhere else will now be delivered to the correct URL.

Note that the code is very specific -- I will work exactly and only as I described it, and you may need to modify it if my description does not encompass all of the URL corrections you need.

The first line may not be needed. It can cause an error if needed and missing, or if not needed and present.

If you already have other working rewriterules in your .htaccess file, neither of the first two lines will be required.

For more information, see the documents cited in our forum charter [webmasterworld.com] and the tutorials in the Apache forum section of the WebmasterWorld library [webmasterworld.com].

Jim

sfseo

2:51 am on Sep 29, 2006 (gmt 0)

10+ Year Member


thanks Jim. I just found out from my Hosting company that we are on a Windows Server and the .htaccess file on our root directory won't work. Is there anything equivalent to the .htaccess file for a Windows server?

jdMorgan

3:21 am on Sep 29, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Windows, huh...

You can use ISAPI Rewrite, which you will have to purchase if your host doesn't provide it. And that's if your host is willing to install it for you.

Jim

sfseo

3:34 am on Sep 29, 2006 (gmt 0)

10+ Year Member


Yup...found out it was $69 bucks =(