jdMorgan - 4:48 am on Oct 9, 2002 (gmt 0)

Natashka,

I've noticed "Scooter" on the list. Isn't the Altavista's crawler called "Scooter"? I remember it from the old good days, before they've re-designed their site and there was even a cartoon picture of that "Scooter" on their site... That was looong time ago though, maybe they've changed the crawler's name.

Yes, that's AltaVista's spider. Some have added access blocks for Scooter/1.0 because it has recently been ignoring robots.txt. AV is apparently using Scooter/1.0 to index graphics. Scooter/3.2 is the one they usually use to index html pages. I suspect someone added "Scooter" to the list without making this distinction.

It does seem like a long time ago, but it wasn't really, when AV enjoyed all the praise (and criticism) that Google enjoys today. That's the reason we should never target just one search engine when working on our sites, even if that search engine drives 90% of our traffic today. Things change.

Jim