Se Friendly URLs and duplicate content?

Can seacrh engine friendly URLs generate duplicate content?

[nfinland]

I noticed Google is picking up my blog search engine friendly URLs and the dynamic ones as well. I was told this could trigger a duplicate content filter. Can I prevent this?

I was given the advice to ask for help here. My knowledge of .htacces files are zero (0).

The only way around it is to use {THE_REQUEST} (Rewrite Condition) to redirect the dynamic URLs to the static location, and then let your rewrite serve the information from the dynamic location back to the static one - gets a little complicated, but can be done. Justin

I use Nucleus and this is how my current .htacces file looks like:

RewriteEngine On
RewriteRule ^archive-([0-9]+)-([0-9]+)-([0-9]+).html+ index.php?archive=$2-$3&blogid=$1
RewriteRule ^item-([0-9]+).html+ index.php?itemid=$1
RewriteRule ^archivelist-([a-z]+).html+ index.php?archivelist=$1

[omoutop]

use robots.txt at the root of your server to disallow bots and spiders from viewing the .php files...

example:
Disallow: reviews3.php
Disallow: /reviews/test.php
Disallow: /guestbook/posts.php

[nfinland]

I want the SE friendly URLs to be indexed so this is not and option.

Here´s an example of a dynamic and a SE friendly URL:

/blog/item-9.html
/blog/index.php?itemid=9

If I disallow /blog/ then I loose all search engine traffic for it.

[jd01]

Glad you made it in here:

/blog/item-9.html
/blog/index.php?itemid=9

The rewrite would be:

RewriteCond %{THE_REQUEST} \?itemid=([0-9]+)
RewriteRule blog/index\.php$ http://yoursite.com/blog/item-%1.html [R=301,L]

Rule: If the URL requested is blog/index.php, check the condition and if the condition pattern matches, redirect to the static page /blog/item-NUMBER.html

Condition: If the URL requested is an original request (anything that is not an internal rewrite EG click on a link, typed in a browser, etc.) and the query string has itemid= any number, continue with the rewrite.

This will rewrite blog/index.php?itemid='1 or more numbers' to item-'the number'.html

If you have more than one variable in the query string, you may need multiple rules and conditions.

If you have more than one php page, you will definitely need more than one.

Hopefully this gives you an idea or two.

Let us know if you need more help.

Justin

Oops! didn't read the entire first post - Don't tell anybody I skim...

# RewriteRule ^archive-([0-9]+)-([0-9]+)-([0-9]+).html+ index.php?archive=$2-$3&blogid=$1

# Converse of the above rule:
RewriteCond %{THE_REQUEST} \?archive=([0-9]+)-([0-9]+)&blogid=([0-9]+)
RewriteRule blog/index\.php$ http://yoursite.com/archive-%1-%2-%3.html [R=301,L]

I'll let you play with the last one.

[edited by: jd01 at 8:57 am (utc) on Sep. 9, 2005]

[omoutop]

jd01's asnwer is another good option (better than mine since no robots.txt is needed)..

However...u mentioned...
///////////////////////////
/blog/item-9.html
/blog/index.php?itemid=9

If I disallow /blog/ then I loose all search engine traffic for it.

////////////////////////////

to loose all trafic to blog u need this:
Disallow: /blog/

to hide the specific file u need without hiding the whole folder u need this:
Disallow: /blog/index.php?itemid=9

this is NOT APPLIED TO THE WHOLE FOLDER....

[nfinland]

Thanks a lot - I will work on this...

[omoutop]

to loose all trafic to blog u need this:
Disallow: /blog/

to hide the specific file u need without hiding the whole folder u need this:
Disallow: /blog/index.php?itemid=9

CORRECTION

NOT Disallow: /blog/index.php?itemid=9

BUT Disallow: /blog/index.php