homepage Welcome to WebmasterWorld Guest from 54.166.159.110
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / WebmasterWorld / New To Web Development
Forum Library, Charter, Moderators: brotherhood of lan & mack

New To Web Development Forum

    
URL path to forum has two slashes
And Googlebot is crawling them
AndyA

5+ Year Member



 
Msg#: 3096938 posted 3:12 am on Sep 26, 2006 (gmt 0)

I have no idea how this happened, but the path to a typical page on my forum is:

http:/ /mydomain.com/forum/index.php?

And Googlebot is crawling URLs that look like this:

http:/ /mydomain.com/forum//index.php?

And my server is giving a 200 OK code for both URLs. I think this might be causing duplicate penalties but I have no idea why my server isn't returning an error. Do I fix this with htaccess?

Edit to add: I did find a slash at the end of the Board URL setup in the admin panel, and removed it. This may remove the double slash from this point forward, but now I need to stop Googlebot from doing it.

[edited by: AndyA at 3:16 am (utc) on Sep. 26, 2006]

 

mariolosek

5+ Year Member



 
Msg#: 3096938 posted 7:26 am on Sep 26, 2006 (gmt 0)

Actually there is no duplicate content inside the same domain name. In this situation Google will index only one url.

You can check your code or do 301 redirection from:

http:/ /mydomain.com/forum//index.php? to

http:/ /mydomain.com/forum/index.php?

in .htaccess

AndyA

5+ Year Member



 
Msg#: 3096938 posted 11:49 am on Sep 26, 2006 (gmt 0)

Wouldn't that code only work for the /index.php page? I've got thousands of pages in my forum, many of which Google is crawling.

I found this code in another thread here at WebmasterWorld:

RewriteCond %{REQUEST_URI} ^(.*)//+(.*)
RewriteRule .* /%1/$2 [R=301,L]

It's supposed to remove the double slashes from deeper folders, but doesn't seem to work on my site.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / WebmasterWorld / New To Web Development
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved