Welcome to WebmasterWorld Guest from 54.160.131.144

Message Too Old, No Replies

301 Redirect - trying to fix root vs. index page problem

     

ramachandra

11:40 am on Mar 20, 2006 (gmt 0)

10+ Year Member



As I have discussed earlier in my other posts, one thing I have got really confused, why my site index page is showing as two different pages as www.mydomain.com and www.mydomain.com/index.asp in Google. I have a 301 redirect code written in index.asp page as below:

<%
PathInfo = Request.ServerVariables("PATH_INFO")
ServerName = Request.ServerVariables("SERVER_NAME")
Iswww = InStr(ServerName,"www.mydomain.com")

If Iswww < 1 Then
NewLocation = "http://www.mydomain.com/"
Response.Status="301 Moved Permanently"
Response.AddHeader "Location", NewLocation
End If
%>

The above code resolves the non-www and www issue. No where in my website pages is pointing to index.asp rather its pointing to root i.e. [mydomain.com...]

I desperately need to know that what makes Google see index page as two different pages?

I sincerely welcome your comments and suggestions.

Thank you

Halfdeck

6:20 am on Mar 21, 2006 (gmt 0)

5+ Year Member



www and non-www is a different issue from / and index.html indexed as separate urls. I'd run Xenu to double check that nothing points to index.asp. Also, someone else linking to /index.asp can cause this problem, so to deal with it, I suggest you write additional code that sets up a 301 from index.asp to /.

g1smd

12:35 am on Mar 22, 2006 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Redirect the index page to "/".

Redirect all non-www to www.

Make sure that the redirect is a 301 redirect. Use WebBug to check it.

ramachandra

5:42 am on Mar 22, 2006 (gmt 0)

10+ Year Member



Thanks Halfdeck and g1smd for your valuable suggestions.

trader

2:18 am on Mar 23, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



...Redirect the index page to "/". Redirect all non-www to www. Make sure that the redirect is a 301 redirect

I know G1smd always strongly recommends (both here and at other forums) you should do the above but I must disagree with G1 on the issue.

My theory has always been if you are listed both with and also without the www that in effect you likely end up listed nicely both ways, and what is wrong with that anyway? Another benefit is if for some odd reason one of them drops off or declines greatly the other listing is still there to carry the ball. Call it an insurance policy if you will

Here are details on a real time happening on one of my sites within the past few 2 weeks. Since the site was in the SERPS both ways site non-www (index page) beat out the former #1 site (who dropped to #2), with my www listing being 3rd with an htm sub-page listed.

My new new #1 position (no www) would not be there at all if not listed without the www, and my #3 position would have in all likelihhod still be #3 anyway. So the net result of having it listed both ways is a #1 ranking, with more traffic than if listed only one way.

Of course, I realize what I am saying is somewhat controversial (especially as far as G1 is concerned) but I really feel I am correct on the issue and have previously seen similar things happen many times, resulting from being listed both ways.

I suggest you do not do the 301 he recommends. If G wants to list you both ways simply let them do so. It is G's decision to list your site both ways. It would seem obvious if they wanted to they could easily program a rule which will list you only one way. The fact they do not do so suggests to me perhaps G sees nothing wrong with it as I do too.

tedster

2:40 am on Mar 23, 2006 (gmt 0)

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member



The issue here is that technically, the content at the domain root is not required to be the same as the content at index.htm or whatever the file name is for the home page. It is not at all a trivial task for a search engine to sort out what to do with two urls that point to what seems, at the moment, to be the same content.

If Google has both urls indexed, then the influence of backlinks (anchor text, PageRank and so on) is possibly being split into two "buckets", according to whichever url the backlink uses. This gives each of the urls more of a challenge in its ability to rank. The 301 redirect would handle the situation without requiring a site owner to get cooperation from every domain that links to them.

Of course, fixing all your own Home Page links to point to the domain root rather than a path that includes the file name is a good practice and can often result in better search results. In fact, I find it's also a good practice to include the final slash as well -- http://www.example.com/ rather than http://www.example.com.

ramachandra

6:16 am on Mar 23, 2006 (gmt 0)

10+ Year Member



>> Of course, fixing all your own Home Page links to point to the domain root rather than a path that includes the file name is a good practice and can often result in better search results. In fact, I find it's also a good practice to include the final slash as well -- http://www.example.com/ rather than http://www.example.com.

All of my internal pages are pointing to the root page with final slash at the end.

Trader,

If search engine indexes two pages (url) of same content, will that be going to harm and penalised for Duplicate content in SERPs?

As far as I feel, in a regular crawl any one of the page will definately go into supplemental and will be out of SERPs.

g1smd

11:38 am on Mar 23, 2006 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



You'll find that one of the index page versions gets cached less frequently, and eventually will go Supplemental.

It is quite possible for a server to have:
www.domain.com/index.html
www.domain.com/index.htm and
www.domain.com/default.asp all serving different content, and a call for
www.domain.com/ serving the same content as www.domain.com/index.php does.

Search engines treat each URL that they find as being a separate page. They try to reduce the number of listings if two pages are found to be identical. Help them to do their job by presenting only one URL for each piece of content. See my comments about duplicate content at [webmasterworld.com...] too.

trader

2:43 pm on Mar 23, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Sorry G1, I am not at all convinced there is such a thing as the alleged duplicate content penalty and think it could be merely a popular urban legend.

After all, the web is all about duplicated content, right? Without duplicate content the web would be a fraction of its size, popularity and usefullness. A vast amount of published material is syndicated or consists of news media publications which appear on multiple websites and proves valuable due to it being so widespread, and therefore easily found by a websurfer.

Over the yrs I have seen zillions of htm pages which ranked high but upon careful evaluation they are not the original source. How would the SE's easily figure out with accuracy who the original source is so they could go through the trouble of assigning everyone else a penalty? (Even if possible, why would they bother anyway in view of the obvious value of widespread distribution)?

That great difficulty is especially valid if you consider so many sites routinely display their own copyright notice on the webpages, regardless of where the page content came from.

P.S. Of course, I realize your own websites should not carry duplicate content but that is a different context.

Miop

4:52 pm on Mar 23, 2006 (gmt 0)

10+ Year Member



<I suggest you do not do the 301 he recommends. If G wants to list you both ways simply let them do so. It is G's decision to list your site both ways. It would seem obvious if they wanted to they could easily program a rule which will list you only one way. The fact they do not do so suggests to me perhaps G sees nothing wrong with it as I do too. >

My non-www page has not been visited by G since 11/6/05 when G decided to split it from the www version.

Only on little daddy has the redirect I did 4 months ago been picked up - all other DC's are still showing the presence of non-www.

g1smd

4:57 pm on Mar 23, 2006 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



>> I suggest you do not do the 301 he recommends. <<

Matt Cutts has repeatedly posted that you should do the redirect.

trader

6:39 pm on Mar 23, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I follow the 301 htaccess instructions here: webconfs.com/how-to-redirect-a-webpage.php
and put this in my htaccess file: Redirect permanent / [new-urldotcom...]
however, when I go here: webconfs.com/http-header-check.php to validate it reports it is a 302 and not a 301.

The above scenario is ongoing for me for ages and a reason I simply do not bother much with trying to do 301's as it seems they always somehow end up as really being 302's! Anyone know why or how to do it right so it really validates as a 301?

g1smd

6:47 pm on Mar 23, 2006 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Yes. See here: [google.com...]

trader

8:38 pm on Mar 23, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



But I am confused as to why what I am told to do by webconfs.com and others appears to not really be a 301 when validated, even though the code say's Permanent Redirect?

Redirect permanent / [new-urldotcom...]

trader

3:55 pm on Mar 24, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Can someone answer this as to why when my htaccess file say's this:

Redirect permanent / [new-urldotcom...]

I then validate it and am told it really is a 302 and not a 301? This has been a huge issue with me since I started to follow all the good advise from G1 and others to use 301 and not 302.

For some reason I Could not find the answer in the many links given me to other thread search results

sit2510

5:02 pm on Mar 24, 2006 (gmt 0)

10+ Year Member



It is quite possible for a server to have:
www.domain.com/index.html
www.domain.com/index.htm and
www.domain.com/default.asp all serving different content, and a call for
www.domain.com/ serving the same content as www.domain.com/index.php does.

--------------

I would strongly second G1smd's recommendation about 301 redirect of the above issue. Some of my sites got sabotage severely not long ago when Google didn't understand how to resolve the canonical urls of my sites correctly. Just adding the right 301 redirect is an easy task, but risking the sites to be dumped by not putting this 301 redirect because of this silly canonical thing by Google is very unwise and could be costly.

trader

7:50 pm on Mar 24, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Never realized how good G1smd's advice was until recently when I looked up many of his posts both here and elsewhere and see he is well known and extremely knowlegable.

It seems most other experts (including Matt Cutts Blog) suggest the redirect goes to the www but I was wondering why not 301 to the non-www instead? Why go to the www?

That would appear especially good to do since the non- www is better to use as far as Adsense tracking goes since 1 channel for the non-www also records the www traffic but the www channel does not record the non-www? Hope that makes sense?

Also, still waiting for answers as to why the htaccess line works OK but the validation services I use both say it really is a 302, not 301? Am I the only one observing that result perhaps? Is there any point in doing that if it does not correctly validate as a 301?

tedster

8:02 pm on Mar 24, 2006 (gmt 0)

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member



why not 301 to the non-www instead?

Which way to redirect is really just a preference. If you prefer going to the no-www for Adsense tracking or whatever reason, there's no problem. It's just that most people tend to add the "www" no matter what, so redirecting "no-www" to "with-www" seems to be the most common choice. Most incoming links will point to the "with-www" so even if a search engine temporarily munges up 301 redirects, more of your backlinks are covered.

waiting for answers as to why the htaccess line works OK but the validation services I use both say it really is a 302, not 301

What you posted above does not look complete to me, and as you are surmising, you can't say it "works" until the header is actually showing a 301.

You might want to post your question in our Apache Forum [webmasterworld.com] to be sure you're getting the most informed input.

g1smd

8:16 pm on Mar 24, 2006 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



For me this is logical: domain.com is the domain that you registered, and then www.domain.com, mail.domain.com, ftp.domain.com, newinvention.domain.com are the URLs for each of the services on that domain.

There is no reason at all why you can't use domain.com for your website, just as there is no reason at all why would couldn't put the mailserver there instead. Organisationally, it just seems logical to have each service on its own subdomain.

.

As for your failing redirect code, can you try the mod_rewrite code that is posted widely, instead, and see if that works for you?

<Note: for those with trouble yesterday, my PM inbox should be working for me now>

Purposeinc

6:44 am on Apr 11, 2006 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



I just finished reading this thread.

I have finally gotten my own htaccess file doing 301 redirects working.

I do not know how to get the index.html to redirect to the /.

Here is my current htaccess file. And I hope G1 will comment on this for me.
Also any other lines that need to be here to handle any other example.com/folder/ vs. example.com/folder problems are greatly apreciated.

RewriteEngine On
RewriteCond %{HTTP_HOST} ^example\.com
RewriteRule ^(.*)$ http://www.example.com/$1 [R=301,L]

Thanks for all the help!
dk

g1smd

11:17 pm on Apr 11, 2006 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



(Someone else originally posted this a few weeks ago)

.

One way catches all subdirectory index files too:

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /(([^/]+/)*)index\.html\ HTTP/
RewriteRule index\.html$
http://www.example.com/%1 [R=301,L]

The [A-Z]{3,9} pattern is intended to match the GET, POST, PROPFIND, etc. HTTP methods. {THE_REQUEST} is the exact request header sent by the browser, for example:

GET /forum/index.php?t=1302 HTTP/1.1

This is exactly as a request would appear in your raw server access log.

The shortest valid HTTP method is three characters long, and the longest is 9.

Purposeinc

5:02 am on Apr 12, 2006 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



Thanks G,

Could you break down what each part of that does, as it is a bit above my head.

If I am understanding it right, that code directs any requests for any specific pages to http://www.example.com,

So that if someone entered
http://example.com/subdirectory/file.html
they would end up at http://www.example.com

and does not direct say
http://example.com/subdirectory/file.html to http://www.example.com/subdirectory/file.html

Am I getting this? or did I totally miss the point?

Thanks!

dk

g1smd

6:34 pm on Apr 12, 2006 (gmt 0)

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



No, I believe that it knocks the filename off the end and retains the directory name.

Asia_Expat

1:06 am on Apr 21, 2006 (gmt 0)

5+ Year Member



I'm completely confused by this... sorry.

Regarding the dupe content problem of / and index.html

I've decided to go down the 301 route to solve this but with examples such as the one above, should I add that as a htaccess file to each individual directory, or to the root directory (i.e., would doind so be a catch all for all /index.html problem directories below the root?)

Asia_Expat

1:09 am on Apr 21, 2006 (gmt 0)

5+ Year Member



I'm also concerned about infinite loops and opther problems... Is there anything I need to be aware of that I may be doing damage with this?
 

Featured Threads

Hot Threads This Week

Hot Threads This Month