Welcome to WebmasterWorld Guest from

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

New Altavista Behaviour...

libwww-perl/5.46 for av?



11:36 pm on May 30, 2001 (gmt 0)

10+ Year Member

Hello, some altavista came by, and crawled my whole site, well-behaved, about 1 hit per second.

The only thing that struck me is the user-agent, libwww-perl. I would have thought AV would use something more ingeniuos that the perl-www library.
www.altavista.com - - [30/May/2001:22:20:54 +0200] "GET / HTTP/1.0" 200 6168 "-" "libwww-perl/5.46"


11:39 pm on May 30, 2001 (gmt 0)

10+ Year Member

oh, something else: did NOT get the robots.txt file.


11:57 pm on May 30, 2001 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

What was the ip address?


7:44 am on May 31, 2001 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

Sounds like someone spoofing an AV useragent.


11:51 pm on Jun 4, 2001 (gmt 0)

10+ Year Member

The line below the dashes is my actual logfile entry, its all I have (plus a few others with different GET requests)

June 1 to 3, I saw a few scooters coming by,

they exhibited "normal" scooter-behaviour (ie. respected robots.txt, did not put more than one hit per second on my site, etc..)

Was just puzzled that av would use an (unadapted?) libwww-perl


Featured Threads

Hot Threads This Week

Hot Threads This Month