Welcome to WebmasterWorld Guest from 18.104.22.168 , register , free tools , login , search , pro membership , help , library , announcements , recent posts , open posts Become a Pro Member
New Altavista Behaviour... libwww-perl/5.46 for av? skirril msg:398815 11:36 pm on May 30, 2001 (gmt 0) Hello, some altavista came by, and crawled my whole site, well-behaved, about 1 hit per second.
The only thing that struck me is the user-agent, libwww-perl. I would have thought AV would use something more ingeniuos that the perl-www library.
-- www.altavista.com - - [30/May/2001:22:20:54 +0200] "GET / HTTP/1.0" 200 6168 "-" "libwww-perl/5.46"
skirril msg:398816 11:39 pm on May 30, 2001 (gmt 0)
oh, something else: did NOT get the robots.txt file. Brett_Tabke msg:398817 11:57 pm on May 30, 2001 (gmt 0)
What was the ip address? WebGuerrilla msg:398818 7:44 am on May 31, 2001 (gmt 0)
Sounds like someone spoofing an AV useragent. skirril msg:398819 11:51 pm on Jun 4, 2001 (gmt 0)
The line below the dashes is my actual logfile entry, its all I have (plus a few others with different GET requests)
June 1 to 3, I saw a few scooters coming by,
they exhibited "normal" scooter-behaviour (ie. respected robots.txt, did not put more than one hit per second on my site, etc..)
Was just puzzled that av would use an (unadapted?) libwww-perl