Welcome to WebmasterWorld Guest from

Forum Moderators: mademetop

Message Too Old, No Replies


8:56 pm on Jan 16, 2001 (gmt 0)

Junior Member

10+ Year Member

joined:Aug 1, 2000
votes: 0

After reviewing many logs for many domains, it appears that when search engines do random and deep crawls, they look for the file index.html first.

Is that correct?
If so, then web sites that use home pages like index.cfm, default.htm, index.htm, and others are at a disadvantage when it comes to some spidering.

9:13 pm on Jan 16, 2001 (gmt 0)

Senior Member

WebmasterWorld Senior Member rcjordan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Apr 22, 2000
votes: 0

>are at a disadvantage

I'll add index.shtml to that list.

Also, if the index.extensions are coded, i.e. "mydomain.com/index.shtml" vs. "mydomain.com/," wouldn't this remove any negative impact?

2:21 am on Jan 17, 2001 (gmt 0)

Junior Member

10+ Year Member

joined:Dec 19, 2000
votes: 0

Apache (and IIS, I am sure) let you specify what file the
resloves to.

For Apche it defaults to index.html

4:20 am on Jan 17, 2001 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Aug 24, 2000
votes: 5

A server can (may) have a default to its core directory listing, it could be set to ralph.html!
Old Web History - index was just that, a index of many files, not the html we know today.
It makes sense (to me) that index is still the best (safe) choice, when possible.



9:44 pm on Jan 18, 2001 (gmt 0)

Inactive Member
Account Expired


I used to overcome this problem by posting a copy of my web page at both index.htm and index.html. Problem was it added a bunch of places to change every time I changed my main page. I may be leading you down the wrong path here, but I waited until the spiders had gotten both index.htm and .html, and then set html as a forwarder to .htm. That way I was beyond getting penalized for the forwarder, and still had my sites linked.

Now I get zero robot traffic on .html, as it all directs to .htm.

Be aware that those of us using frames have a problem with this on our no frames pages as well. I don't request people download a new version of their browser, and jsut have a copy of my mainpage there. I found I immediately received more traffic with this one little correction.

Anybody who knows if my self-styled fixes change anything, I wouldn't mind knowing myself.

1:24 pm on Feb 9, 2001 (gmt 0)

Preferred Member from GB 

10+ Year Member

joined:Feb 8, 2001
votes: 0

I have a site that currently ranks number 1 in AV.com and Google on competitive keywords, and uses index.shtml so it can't all be bad! That's all I know!