Forum Moderators: open
64.124.122.228 - - [24/Jun/2006:08:37:18 -0400] "GET /robots.txt HTTP/1.0" 200 2966 "-" "WebarooBot (Webaroo Bot; [64.124.122.252...]
Seems we've seen this one before: [webmasterworld.com...]
>>
RufusBot
Why are we crawling?
We crawl the web towards the goal of developing a new kind of index/search tool that will bring substantial and previously unavailable exposure to websites...
<<
Exposure, indeed. "Rufus Web Miner" cum "Webaroo Bot" is into some serious mining:
Webaroo - copying the internet to your hard disk
Offline browsing webaroo - copying the internet to your hard disk.
[webmasterworld.com...]
See also: [webaroo.com...]
2.) FWIW:
Last year (once) --
64.124.122.247.webaroo.com
/robots.txt
"RufusBot (Rufus Web Miner; [64.124.122.252...]
This month (multiple) --
64.124.122.228.gw.xigs.net
/robots.txt
"WebarooBot (Webaroo Bot; [64.124.122.252...]
3.) They're also iffy re the IPs associated with the crawling/mining Host and UA string. Courtesy of [dnsstuff.com...] --
IP address: 64.124.122.228
Reverse DNS: 64.124.122.228.gw.xigs.net
Reverse DNS authenticity: [Could be forged: hostname 64.124.122.228.gw.xigs.net. does not exist]
IP address: 64.124.122.252
Reverse DNS: 64.124.122.252.webaroo.com
Reverse DNS authenticity: [Could be forged: hostname 64.124.122.252.webaroo.com. does not exist]
4.) Seeing as how the new name/theme/icon/logo is a kangaroo (sure beats "RufusBot"), I'm happy to tell it to go jump. Again.