Welcome to WebmasterWorld Guest from 54.144.243.34

Forum Moderators: Ocean10000 & incrediBILL

New Altavista Behaviour...

libwww-perl/5.46 for av?

   
11:36 pm on May 30, 2001 (gmt 0)

10+ Year Member



Hello, some altavista came by, and crawled my whole site, well-behaved, about 1 hit per second.

The only thing that struck me is the user-agent, libwww-perl. I would have thought AV would use something more ingeniuos that the perl-www library.
--
www.altavista.com - - [30/May/2001:22:20:54 +0200] "GET / HTTP/1.0" 200 6168 "-" "libwww-perl/5.46"

11:39 pm on May 30, 2001 (gmt 0)

10+ Year Member



oh, something else: did NOT get the robots.txt file.
11:57 pm on May 30, 2001 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



What was the ip address?
7:44 am on May 31, 2001 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Sounds like someone spoofing an AV useragent.
11:51 pm on Jun 4, 2001 (gmt 0)

10+ Year Member



The line below the dashes is my actual logfile entry, its all I have (plus a few others with different GET requests)

June 1 to 3, I saw a few scooters coming by,
scooter3-scooter6.sv.av.com

they exhibited "normal" scooter-behaviour (ie. respected robots.txt, did not put more than one hit per second on my site, etc..)

Was just puzzled that av would use an (unadapted?) libwww-perl

 

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month