Welcome to WebmasterWorld Guest from

Forum Moderators: mademetop

Message Too Old, No Replies

SEs alive and well

spider feeding frenzy

5:07 pm on Jul 6, 2000 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 10, 2000
votes: 0

wow - what a night. I haven't been spidered this hard in long time. Got hit by t-Rex about a dozen times and scooter, slurp, googlebot and Infoseek stopped by to say hi as well as a handful of others. Here's hoping it all bodes well :)

Edited by: oilman

12:14 am on July 7, 2000 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
votes: 111

Slurp has been going nuts on me everywhere. Spidering everything in sight - even form submitted cgi's. It walked through and pulled everypage available (3000) on another site of mine. Trex has been doing the trex thing, but where is that data going to? Infoseek appears to be in link check only mode. Archie is still as aggressive at link checking and spidering as always - just wish some pages would show up over there.