Welcome to WebmasterWorld Guest from

Forum Moderators: mademetop

Message Too Old, No Replies

SEs alive and well

spider feeding frenzy



5:07 pm on Jul 6, 2000 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

wow - what a night. I haven't been spidered this hard in long time. Got hit by t-Rex about a dozen times and scooter, slurp, googlebot and Infoseek stopped by to say hi as well as a handful of others. Here's hoping it all bodes well :)

Edited by: oilman


12:14 am on Jul 7, 2000 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

Slurp has been going nuts on me everywhere. Spidering everything in sight - even form submitted cgi's. It walked through and pulled everypage available (3000) on another site of mine. Trex has been doing the trex thing, but where is that data going to? Infoseek appears to be in link check only mode. Archie is still as aggressive at link checking and spidering as always - just wish some pages would show up over there.

Featured Threads

Hot Threads This Week

Hot Threads This Month