| Welcome to WebmasterWorld Guest from 126.96.36.199 |
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
|Subscribe to WebmasterWorld|
|Slurp/cat and Slurp/si|
What's the difference?
| 3:01 pm on Jul 11, 2003 (gmt 0)|
Noticed both crawling my site yesterday.
| 4:34 pm on Jul 11, 2003 (gmt 0)|
Slurp/si just reads robots.txt, and Slurp/cat does everything else.
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
© Webmaster World 1996-2013 all rights reserved