coopster
msg:3643999 | 4:41 pm on May 7, 2008 (gmt 0) |
No such thing as a stupid question, my friend! Near the top of the page for every forum here at WebmasterWorld you will find some links to the Forum Charter and the Forum Library that you are viewing. This forum is no different and you will find quite a bit of introductory information regarding robots.txt in the robots.txt forum Charter [webmasterworld.com]
|
pageoneresults
msg:3644002 | 4:45 pm on May 7, 2008 (gmt 0) |
These days I might be a bit leery about using my robots.txt for too much. No, I'm more apt to do something a bit more technical and not put a directory list in there for prying eyes to see. Especially Googlebot and it's damn URI only listings for references discovered via the robots.txt file.
|
|
