| Welcome to WebmasterWorld Guest from 126.96.36.199 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Become a Pro Member|
| 12:53 pm on Apr 18, 2000 (gmt 0)|
Spider has been picking up +/- 30 pages a day. What is its purpose?
| 9:04 am on Apr 19, 2000 (gmt 0)|
The good old proxy cache spider. Grabs pages for aol users and then serves them out of the AOL cache. It runs with many different user agents for that many different AOL users.
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved