MIIxpc?

[mark_roach]

I don't currently block anyone. However after blowing my bandwidth limit last month I intend to start doing so now.

Does anyone know why I shouldn't add these 3 to the list ?

JennyBot
MIIxpc
teoma_agent3

[Machiavelli]

User-agent: Googlebot
Disallow: /

[mivox]

Why would you want to ban Google from your entire site?

[Air]

>MIIxpc

Not absolutely sure, but there is some indication that this may be altavista.nl or altavista.de, can anyone confirm?

[oLeon]

Air,
I notice that spider, too. I donīt know where it comes from, but I would guess it spiders the livesearches of another engines.

[volatilegx]

Isn't this for the robots.txt file? Am I missing something? Most of these robots would never check the robots.txt file, right?

Dan

[mark_roach]

>MIIxpc

>I notice that spider, too. I donīt know where it comes >from, but I would guess it spiders the livesearches of >another engines.

oLeon, do you think that is what is going on here ?

195.121.6.106 - - [28/Mar/2001:06:35:55 -0500] "GET / HTTP/1.1" 200 9693 "http://195.121.7.86/cgi-bin/zoeken/avsearch.cgi?pg=q&q=border+terrier&kl=XX&what=web&stq=10" "Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt)"

195.121.6.106 - - [28/Mar/2001:06:35:59 -0500] "GET /images/film.jpg HTTP/1.1" 200 5911 "http://www.champdogs.co.uk/html/home.html" "Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt)"

212.78.177.71 - - [28/Mar/2001:06:36:00 -0500] "GET /images/film.jpg HTTP/1.0" 200 5911 "-" "MIIxpc/4.2"

195.121.6.106 - - [28/Mar/2001:06:36:12 -0500] "GET /html/search_menu.htm HTTP/1.1" 200 1815 "-" "Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt)"

195.121.6.106 - - [28/Mar/2001:06:36:12 -0500] "GET /html/search.htm HTTP/1.1" 200 824 "http://www.champdogs.co.uk/html/master_menu.htm" "Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt)"

212.78.177.71 - - [28/Mar/2001:06:36:12 -0500] "GET /html/search.htm HTTP/1.0" 200 824 "-" "MIIxpc/4.2"

212.78.177.70 - - [28/Mar/2001:06:36:12 -0500] "GET /html/search_menu.htm HTTP/1.0" 200 1815 "-" "MIIxpc/4.2"

It followed the surfer right round my site, taking the identical pages including the graphics.

[sjoerd]

It's an accelerator of Mirrorimage.net. Just resolve the IP and then do a whois on the domain xpc-mii.net and you'll end up at mirrorimage.net.
They offer some kind of shared proxy-cache. Everytime you get visited by websurfer having enabled a proxy that is using this shared proxy-cache, you'll find this thing hitting your site some 30 minutes later...

[max_b]

Is it supposed to crawl behind a .htaccess fence?

On my site it also follows the behaviour of another user.

[skirril]

To my knowledge, it is very hard to crawl behind such a fence, because its the server blocking. Unless of course the fence had holes in it..