New bot? WIRE/0.1

[dhatz]

I have this one downloading thousands of pages across several websites and domains.

It did download robots.txt before going ahead, but it has no URL or other contact info, and I can't seem to find more about it at cluster.ischool.washington.edu which is who the IPs belong to.

I decided to deny access to it, until I learn more about it. This is from the logs (just one site, it's accessing several sites on different IPs, so it must have ample bandwidth, or really-really like me ;-)

128.X08.131.220 - - [07/May/2004:18:54:45 +0300] "GET /****.html HTTP/1.1" 403 235 "-" "WIRE/0.1 (Linux; i686; Bot,Robot,Spider,Crawler)"
128.X08.131.220 - - [07/May/2004:18:54:50 +0300] "GET /****.html HTTP/1.1" 403 235 "-" "WIRE/0.1 (Linux; i686; Bot,Robot,Spider,Crawler)"

[edited by: DaveAtIFG at 7:40 pm (utc) on May 7, 2004]
[edit reason] Obscured IP [/edit]

[Sanenet]

Go to the IP (128.X08.131.220).

[edited by: DaveAtIFG at 7:41 pm (utc) on May 7, 2004]
[edit reason] Obscured IP [/edit]

[dhatz]

So were you able to find anything about what they plan to do with that data?

Their bot has already downloaded over 10.000 documents from several Websites of mine, on different IPs, hosted in Greece and in US, in 6 different languages.

What's the point?

I found about WIRE bot here:

[cwr.cl...]

And I guess those people at UWashington have lots of spare bandwidth to waste...

[Sanenet]

It seems to be just an app written by some university students from Chile. If it's annoying you, ban it. Or, if it's disobeying orders, send them an email telling them of it's bad behaviour. These guys love feedback!

[Yidaki]

>These guys love feedback

Yep, robot programmers that don't hide normally appreciate feedback and need feedback to work on improvements and to repair bugs.

(Due to the feedback i sent to the ibm / almaden developer team a few years ago, their bot finally learned how to parse robots.txt files that are macintosh formatted - non standard cr / lf.)

[volatilegx]

So we've concluded that WIRE is an experimental spider not currently linked to any live search engine?

[Yidaki]

>o we've concluded that WIRE is an experimental spider

Yep. The software is free and available at the Center for Web Research University Of Chile. The requests are obviously coming from the University of Washington Information School where they run several projects related to web search. The most related project is the The Web Tango Project [webtango.ischool.washington.edu], a research project including Designing interfaces for enhancing Internet searching ...

[sidyadav]

I love these kind of things. Always fun to download, install, run.

WIRE produced some excellent results. I wonder if a company/person can actually use it in a commercial project.

Sid