homepage Welcome to WebmasterWorld Guest from 54.234.141.47
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

    
Two URLs access the same page - how to fix
shaunm



 
Msg#: 4487978 posted 1:45 pm on Aug 24, 2012 (gmt 0)

Hi,

Some of my website's pages are accessible in two ways
Such as
example.com/product-overview.aspx
example.com/product-overview

Will there be any problem because of this? Are they considered 2 entirely different pages even though they are same by SEs?

But only one page is showing up in Google's index which is with '.aspx'


What should I do in this regard? Should I put a redirect in this place?


Thanks a lot!

 

FranticFish

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 4487978 posted 3:57 pm on Aug 24, 2012 (gmt 0)

I don't know of any way that you can redirect your .aspx url to your extensionless one. The server knows that the real page is the .aspx version, and it displays that when asked for the extensionless one. So if you then try to redirect back to the extensionless display url you end up in a loop.

I'd...
- change the .aspx version file name (product-overview2.aspx will do),
- get your url rewriting to look that up instead when product-overview is requested, then
- remove product-overview.aspx from Google's index by blocking it in robots.txt then using the removal tool in WMT

Perhaps you could specify a canonical of the extensionless version as an alternate fix but I don't know how reliable that is having not tried it myself.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4487978 posted 4:42 pm on Aug 24, 2012 (gmt 0)

I don't know of any way that you can redirect your .aspx url to your extensionless one. The server knows that the real page is the .aspx version, and it displays that when asked for the extensionless one. So if you then try to redirect back to the extensionless display url you end up in a loop.

The server doesn't know anything. Always remember that. Computers Are Stupid. They just happen to be stupid very, very, very fast.

Yes, google thinks "page" and "page.xtn" are two different things, just as it thinks "directory/" and "directory/index.html" or "www.example.com/filename" and "example.com/filename" are different things.

What should I do in this regard? Should I put a redirect in this place?

YES. Stroll over to the Apache forum and look for a recurring piece of boilerplate about the redirect-to-rewrite two-step.

Short version:

Use mod_rewrite. Or, ahem, IIS equivalent. Someone can translate, if necessary.

REDIRECT unwanted ugly URL to desirable pretty URL, with accompanying RewriteCond looking at {THE_REQUEST}. You only want to redirect the requests that came in from outside, not the ones that result from an internal rewrite. That's how you avoid your infinite loop.

REWRITE pretty URL to real home of content-- which might be the identical address you only just finished redirecting from :)

shaunm



 
Msg#: 4487978 posted 7:29 pm on Aug 24, 2012 (gmt 0)

@FranticFish

I don't know of any way that you can redirect your .aspx url to your extensionless one. The server knows that the real page is the .aspx version, and it displays that when asked for the extensionless one. So if you then try to redirect back to the extensionless display url you end up in a loop.


Thank you for answering. And I am sorry that if my question have some misunderstandings. Btw, I never wanted the '.aspx' version to redirect to extentionless. And you are wrong the server displays both version when requested.

I actually wanted to ask if it is correct to redirect the extensionless to extension one if there is a problem with two versions of the same page. Also, I strongly disagree on your first sentence, yes it can surely be done!

Thanks

shaunm



 
Msg#: 4487978 posted 7:32 pm on Aug 24, 2012 (gmt 0)

@lucy24

Thank you so much. All I wanted to know is that whether redirecting the extentionless to .aspx will prevent my site from duplicate content or what.

At the same time, I wonder why only the .aspx version is indexed by Google not the one without .aspx. Server displays both version of pages when requested, though.

And I am very much familiar with REWRITE rules :)

Once again, thanks

Sand



 
Msg#: 4487978 posted 8:09 pm on Aug 24, 2012 (gmt 0)

Google is pretty forgiving when it comes to file extensions these days. They've pretty much got it figured out at this point, but to be safe, it's a good idea to pick one and redirect the other to it. It won't do any harm, and can only help.

As for why only one is indexed right now: it could be any number of things (such as how you're linking to the pages internally), but in the end, Google probably doesn't want to have two pages in its index from the same site that are identical. They know what's going on with the file extension, and have made the choice for you as far as which one they're going to show.

aakk9999

WebmasterWorld Administrator 5+ Year Member



 
Msg#: 4487978 posted 8:18 pm on Aug 24, 2012 (gmt 0)

At the same time, I wonder why only the .aspx version is indexed by Google not the one without .aspx. Server displays both version of pages when requested, though.


Unless you are using canonical link element, Google has probably decided to pick one version of URL and shows that one.

BTW, it could be someting in your IIS server setup that says "if I get a URL without the extension, append .aspx and forward this to this script".

Or something as above is possibly handled within the custom Rewrite module.

This would result in the exact case you are having: the .aspx page would generate the page content and return it to the client, whilst the client's URL would not change (would remain extensionless).

I seem to remember that IIS 7 for example has Rewrite manager where you can set up rules similar to the above - it is worth checking.

g1smd

WebmasterWorld Senior Member g1smd us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4487978 posted 9:09 pm on Aug 24, 2012 (gmt 0)

In general, when two different "path and file"s are available to show the same content, the shorter one should be the canonical URL.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4487978 posted 10:03 pm on Aug 24, 2012 (gmt 0)

Google probably doesn't want to have two pages in its index from the same site that are identical

Unless the site name contains an element like "amazon" or "wiki", in which case they will happily show five marginally different versions of the same page, one after the other ;)

shaunm



 
Msg#: 4487978 posted 6:12 am on Aug 27, 2012 (gmt 0)

@lucy24
Unless the site name contains an element like "amazon" or "wiki", in which case they will happily show five marginally different versions of the same page, one after the other ;)


I often come across search results like this. What could be the main reason that such websites occupy more than 5 continuous placement/ranking for a single search query?

As far as I know, it only happens with e-commerce websites no matter big or small and also with directories as well.

Thanks,

shaunm



 
Msg#: 4487978 posted 6:38 am on Aug 27, 2012 (gmt 0)

@g1smd
In general, when two different "path and file"s are available to show the same content, the shorter one should be the canonical URL.

Thanks. But in my case, it's the opposite :)

@Sand
Thanks a lot!

@aakk9999
Thanks for answering!
it could be someting in your IIS server setup that says "if I get a URL without the extension, append .aspx and forward this to this script"

Can you please tell me what you meant by the above?


Thank you all!

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4487978 posted 9:21 am on Aug 27, 2012 (gmt 0)

Oh, are you on IIS? Oops. I'd assumed Apache, and was wondering if mod_negotiation [httpd.apache.org] was being excessively helpful. That is: in a world without search engines or canonicalization it would be nice for the server to read the user's mind and say "when they asked for 'foo' they must have meant 'foo.html' because there's no /foo/ directory". Unlike your average Apache question, MultiViews only comes up about twice a year ;)

aakk9999

WebmasterWorld Administrator 5+ Year Member



 
Msg#: 4487978 posted 12:09 pm on Aug 27, 2012 (gmt 0)

There is Rewrite module on the IIS server that can be accessed via server admin. Certain behaviour (rules) can be set up there. Rules may use regular expressions. So for example, there could be a rule saying that if Request URL is extensionless, to forward it to the script of the same name with .aspx extension.

This is forwarding and not redirect, this would act similar to server.transfer. I am not saying this is happening, but it is worth checking.

shaunm



 
Msg#: 4487978 posted 12:15 pm on Aug 27, 2012 (gmt 0)

@aakk9999
Thanks :)

I have access to the Rewrite module.
So for example, there could be a rule saying that if Request URL is extensionless, to forward it to the script of the same name with .aspx extension.

Does that mean the one with extension and the one without extension are both operated by this command? So that I am getting the same page when requested from example.com/text.aspx and example.com/text?

I should seriously check that in that case, thanks for telling me about server forwards. I only knew about redirects.

shaunm



 
Msg#: 4487978 posted 12:21 pm on Aug 27, 2012 (gmt 0)

@lucy24
Seriously, I don't understand what you were trying to say. Can you please explain?

Also, can you please tell me why pages of the same website are ranking in the first page of search results for a keyword and without any break? Such as ranking in 1st, 2nd, 3r, 4th?

Thanks,

netmeg

WebmasterWorld Senior Member netmeg us a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month



 
Msg#: 4487978 posted 12:31 pm on Aug 27, 2012 (gmt 0)

Also, can you please tell me why pages of the same website are ranking in the first page of search results for a keyword and without any break? Such as ranking in 1st, 2nd, 3r, 4th?


That's a completely different topic than that of this post, and there are already a couple of posts in this forum on that topic. Start here:

[webmasterworld.com...]

shaunm



 
Msg#: 4487978 posted 12:39 pm on Aug 27, 2012 (gmt 0)

@netmeg
Thanks for that :)
I know it's completely off topic, but wanted to ask since lucy24 mentioned about that.


Cheers!

g1smd

WebmasterWorld Senior Member g1smd us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4487978 posted 6:42 pm on Aug 27, 2012 (gmt 0)

In general, when two different "path and file"s are available to show the same content, the shorter one should be the canonical URL.
Thanks. But in my case, it's the opposite


Ah, you're mixing up "paths" and "files". Paths are used "out there" on the web. Files are used "here" inside the server.

You have a "file" called "/page.aspx" and while this can be accessed by the obvious "URL" "example.com/page.aspx" if your site is intended to use extensionless URLs the canonical URL will be just "example.com/page".

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4487978 posted 9:08 pm on Aug 27, 2012 (gmt 0)

I don't understand what you were trying to say. Can you please explain?

I think it's the same thing aakk said, only we're speaking different languages ;)

aakk9999

WebmasterWorld Administrator 5+ Year Member



 
Msg#: 4487978 posted 10:44 pm on Aug 27, 2012 (gmt 0)

@lucy
Yes, we were on the same page!

@shaunm
Does that mean the one with extension and the one without extension are both operated by this command? So that I am getting the same page when requested from example.com/text.aspx and example.com/text?

I do not know how your server is set up. But, from your opening post:
Some of my website's pages are accessible in two ways
Such as
example.com/product-overview.aspx
example.com/product-overview

Providing that my understanding is correct, that is, when you enter either of URLs above, the same page content displays and the URL in the address bar shows the URL you have entered, then this would indicate that *somehow* server is set up to internally pass the request to .aspx script.

Now this could happen in several ways:
- Rewrite manager has an entry set up to forward extensionless scripts to the script of the same name with .aspx extension. This means there is a rule set up within Rewrite manager
- there is a "IIS Managed handler mapping" set up to process extensionless requests as .aspx where .aspx variant of the request exists. Handlers map to resources on the Web server. (google "Handler Mappings in IIS 7" for details)
- there is a custom rewrite module in existance that forwards extensionless URLs to .aspx version using the command Server.Transfer (which is used to transfer the execution to a specified script)
- or you may be using ISAPI Rewrite that has this rule somewhere

You also said it happens "for some of your webpages" which to me would indicate there is some custom rule in place if this is not happening for all .aspx pages.

I think you should firstly understand why is this happening before implementing 301 Redirect.

As for the which way redirect should go, whilst extensionless URLs are better (shorter, and technology agnostic), you should also take into the account which format your external links go to, how do you link internally and which URL format Google has in their index.

shaunm



 
Msg#: 4487978 posted 5:06 pm on Aug 29, 2012 (gmt 0)

@g1smd
Thanks for that. It was not intended to use the extension-less URLs. So I am just going to make the one with .aspx as the canonical :)

@lucy24
I should get used to your language then, thanks ;)

shaunm



 
Msg#: 4487978 posted 6:03 pm on Aug 29, 2012 (gmt 0)

@aakk9999
Thank you so much for taking your time to explain it to me! It was big, and I very much appreciate your help :)

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved