Scooter thru all our domains, most pages thus far!

Forum Moderators: open

Message Too Old, No Replies

Scooter thru all our domains, most pages thus far!

pete

1:32 pm on Feb 1, 2001 (gmt 0)

This guy: Scooter-tv36_Mc_kalyan_27

Any news on whats happening - havent seen scooter deep crawl our sites for such a long time now!

msgraph

1:40 pm on Feb 1, 2001 (gmt 0)

Trying to figure out myself.

[webmasterworld.com...]

I found like 20+ versions of this spider. Right now I have it showing up as a Mercator spider on stats to seperate it from the AV spiders. At least until I learn more

luckynh

1:47 pm on Feb 1, 2001 (gmt 0)

here's a new one all over my logs?

209.73.164.42 - GET /robots.txt -HTTP/1.0 Scooter-test-scooter_Mc_kalyan_42

Froggyman

2:12 pm on Feb 1, 2001 (gmt 0)

Never seen Scooter so energetic.

Same IP as others have reported but different UA

UA: Scooter-scooter_Mc_kalyan_43
IP: 209.73.164.40 (scooter.sv.av.com)

mivox

1:28 am on Feb 7, 2001 (gmt 0)

OK I've seen tv/Mc_kaylan/scooter spiders all over the place...

Don't suppose anyone's figured out if they're preparing a huge, comprehensive, well-worth-the-wait update to their DB, or what?

I can't see the point of sending all those spiders out if they're gong to continue letting their search results stagnate.

JuniorHarris

5:30 am on Feb 14, 2001 (gmt 0)

I'm guessing AV parked Scooter for a couple of months, either to tune it and/or in anticipation of LookSmart. Now it appears all those long lost submitted URLs are now being requested and indexed

I keep very detailed logs of all submissions as well as detailed logs of all site activity. After reviewing all logs etc, I can safely say (93.2% certain Regis), that AV is processing a backlog of URL submissions since early December.

The recent index update on AV contains pages that were submitted DEC-4 thru DEC-6 (using a web multi-submission tool). Scooter (kalyan) accessed/indexed those pages (other then the instant verify), on FEB-1 and FEB-2. These pages are now in the current index. I have other URLs in the queue so I should be able to follow up with this theory. The next by date submission was JAN-7.

Brett_Tabke

7:57 am on Feb 14, 2001 (gmt 0)

A couple of rumors floating around and getting repeated is that AV is going to farm out their spidering to a subdivision. The same group that is responsible for developing their enterprise level (corporate) search engine. They also have been tuning mercator for that duty.

Sorry, but I think it is all just testing and research going on.