Welcome to WebmasterWorld Guest from 50.16.78.128

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

New AV spider - Scooter-3.0.g0-test

   
12:18 pm on Mar 27, 2001 (gmt 0)

10+ Year Member



Got this one in my logs yesterday:

209.73.164.127 - - [26/Mar/2001:22:08:42 -0500] "GET /robots.txt HTTP/1.0" 200 616 "-" "Scooter-3.0.g0-test"

12:38 pm on Mar 27, 2001 (gmt 0)

WebmasterWorld Senior Member marcia is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Just a little while ago:

209.73.164.132 - - [27/Mar/2001:06:03:36 -0500] "GET /robots.txt HTTP/1.0" 404 216 "-" "Scooter-3.0.d0"
209.73.164.132 - - [27/Mar/2001:06:03:36 -0500] "GET / HTTP/1.0" 200 11100 "-" "Scooter-3.0.d0"

3:29 pm on Mar 27, 2001 (gmt 0)

10+ Year Member



Got it too

2001-03-27 02:13:30 209.73.164.132 Scooter-3.0.d0 -

2001-03-27 02:13:33 209.73.164.132 Scooter-3.0.d0 -

2001-03-27 04:12:09 209.73.164.132 Scooter-3.0.d0 -

19 pages all edited within the last month. All "level one" pages (links on index)

2001-03-27 06:26:33 209.73.164.132 Scooter-3.0.d0 -

2001-03-27 18:20:33 209.73.164.132 Scooter-3.0.d0 -

11 pages all edited within the last month, some dublicate crawls, also "level 2 and 3"

2001-03-27 18:24:19 209.73.164.132 Scooter-3.0.d0 -

21 pages, mostly non previously spidered pages, all mostly not edited for some time, level 2 and 3.

The site is about 3 years old and has good linkpop, most pages in the first "big" crawl (the newly edited pages) have been submitted to AV.com, AV.europe and AV.asia within the last month.

Well I don't complain :)

First big hug from Scooter in a looong time :))

I'll let you know when and where the pages are indexed, my guess is that Scooter 3 is European - well, a few of them actually are indexed in edited versions in av.com.

But the big news (for me atleast) is that Scooter crawls again ! Hope he hits your place too :)

Message has been updated.

Edited by: ulstrup

7:25 pm on Mar 27, 2001 (gmt 0)

WebmasterWorld Senior Member littleman is a WebmasterWorld Top Contributor of All Time 10+ Year Member



I've also been hit by it lightly.
8:06 am on Mar 28, 2001 (gmt 0)

10+ Year Member



Deep Crawl yes it is :)

2001-03-28 03:46:15 209.73.164.126 Scooter-3.0.g0-11 -
2001-03-28 03:50:19 209.73.164.127 Scooter-3.0.g12-23 -
2001-03-28 04:15:46 209.73.164.129 Scooter-3.0.g24-35 -

These 3 Scooters hit randomly 23 pages in total...

12:19 pm on Mar 28, 2001 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



The question is if this is a data mining crawl or the real deal.
12:14 am on Mar 30, 2001 (gmt 0)

10+ Year Member



my feeling is it maybe the real thing, data mining is usually more extensive and goes into all domains, this crawl only reqests pages from domains not previously banned by AV
9:56 pm on Mar 31, 2001 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Whoa!

I'd given up on AV and paid no attention to it recently, but after reading this thread I ran a check and scooter has been walking all over my site all month!

3/1 to 3/17 scooter crawled ALL the out-of-date pages I've been trying to get removed for quite a while.

3/21 to present scooter has crawled about 250 pages.

I guess when you act like you really don't care, it just drives these spiders wild!