Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Spider.txt


mivox - 10:47 pm on Jan 24, 2001 (gmt 0)


I personally try to keep my websites structured with no more than 1 directory level down from root:

mysite.com/directory/file.htm

The only time I use second level subdirectories is to hold administrative or data/graphics files:

mysite.com/images/icons/icn.gif
mysite.com/images/headers/head.gif

I disallow everything from my image and admin directories anyway, so that keeps everything else nice and clean.... either I allow the spiders into the directory or not.

I'm lazy too... I hate having to redesign and reorganize things once I build them! A lesson for the lazy: it's worth spending the extra time to plan out a site structure at first that will allow you to be lazy later! ;)


Thread source:: http://www.webmasterworld.com/robots_txt/155.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com