Forum Moderators: open

Message Too Old, No Replies

crawler.html/crawlers.html? // How to use them?

Question about use of "crawler" pages

         

WP1

1:45 pm on Apr 21, 2001 (gmt 0)



Hello,

Where can I get some info on "crawler.html" and/or "crawlers.html".

1. What exactly are they??
2. Is it cool to use them?? (that is, do you get penalized by Search Engines for using them?)
3. Can it be used with "robots.txt" and "<META> tags??

Any and all information appreciated.

WP