Welcome to WebmasterWorld Guest from

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies


12:53 pm on Apr 18, 2000 (gmt 0)

Preferred Member

10+ Year Member

joined:Dec 21, 1999
votes: 0

Spider has been picking up +/- 30 pages a day. What is its purpose?
9:04 am on Apr 19, 2000 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
votes: 11

The good old proxy cache spider. Grabs pages for aol users and then serves them out of the AOL cache. It runs with many different user agents for that many different AOL users.

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members