Welcome to WebmasterWorld Guest from 54.224.44.168

Forum Moderators: Ocean10000 & incrediBILL & keyplyr

Message Too Old, No Replies

MIIxpc?

     
9:32 pm on Mar 13, 2001 (gmt 0)

Preferred Member

10+ Year Member

joined:July 28, 2000
posts:580
votes: 0


I don't currently block anyone. However after blowing my bandwidth limit last month I intend to start doing so now.

Does anyone know why I shouldn't add these 3 to the list ?

JennyBot
MIIxpc
teoma_agent3

Machiavelli

10:09 pm on Mar 13, 2001 (gmt 0)

Inactive Member
Account Expired

 
 


User-agent: Googlebot
Disallow: /
10:34 pm on Mar 13, 2001 (gmt 0)

Senior Member

WebmasterWorld Senior Member mivox is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Dec 6, 2000
posts:3928
votes: 0


Why would you want to ban Google from your entire site?

Air

10:01 pm on Mar 15, 2001 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 10, 2000
posts:1253
votes: 0


>MIIxpc

Not absolutely sure, but there is some indication that this may be altavista.nl or altavista.de, can anyone confirm?

3:51 pm on Mar 16, 2001 (gmt 0)

Preferred Member

10+ Year Member

joined:Aug 15, 2000
posts:482
votes: 0


Air,
I notice that spider, too. I donīt know where it comes from, but I would guess it spiders the livesearches of another engines.
11:26 pm on Mar 27, 2001 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 22, 2001
posts:2450
votes: 0


Isn't this for the robots.txt file? Am I missing something? Most of these robots would never check the robots.txt file, right?

Dan

1:42 pm on Mar 29, 2001 (gmt 0)

Preferred Member

10+ Year Member

joined:July 28, 2000
posts:580
votes: 0


>MIIxpc

>I notice that spider, too. I donīt know where it comes >from, but I would guess it spiders the livesearches of >another engines.

oLeon, do you think that is what is going on here ?

195.121.6.106 - - [28/Mar/2001:06:35:55 -0500] "GET / HTTP/1.1" 200 9693 "http://195.121.7.86/cgi-bin/zoeken/avsearch.cgi?pg=q&q=border+terrier&kl=XX&what=web&stq=10" "Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt)"

195.121.6.106 - - [28/Mar/2001:06:35:59 -0500] "GET /images/film.jpg HTTP/1.1" 200 5911 "http://www.champdogs.co.uk/html/home.html" "Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt)"

212.78.177.71 - - [28/Mar/2001:06:36:00 -0500] "GET /images/film.jpg HTTP/1.0" 200 5911 "-" "MIIxpc/4.2"

195.121.6.106 - - [28/Mar/2001:06:36:12 -0500] "GET /html/search_menu.htm HTTP/1.1" 200 1815 "-" "Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt)"

195.121.6.106 - - [28/Mar/2001:06:36:12 -0500] "GET /html/search.htm HTTP/1.1" 200 824 "http://www.champdogs.co.uk/html/master_menu.htm" "Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt)"

212.78.177.71 - - [28/Mar/2001:06:36:12 -0500] "GET /html/search.htm HTTP/1.0" 200 824 "-" "MIIxpc/4.2"

212.78.177.70 - - [28/Mar/2001:06:36:12 -0500] "GET /html/search_menu.htm HTTP/1.0" 200 1815 "-" "MIIxpc/4.2"

It followed the surfer right round my site, taking the identical pages including the graphics.

sjoerd

10:19 pm on Apr 19, 2001 (gmt 0)

Inactive Member
Account Expired

 
 


It's an accelerator of Mirrorimage.net. Just resolve the IP and then do a whois on the domain xpc-mii.net and you'll end up at mirrorimage.net.
They offer some kind of shared proxy-cache. Everytime you get visited by websurfer having enabled a proxy that is using this shared proxy-cache, you'll find this thing hitting your site some 30 minutes later...

max_b

11:59 pm on May 31, 2001 (gmt 0)

Inactive Member
Account Expired

 
 


Is it supposed to crawl behind a .htaccess fence?

On my site it also follows the behaviour of another user.

12:03 am on June 5, 2001 (gmt 0)

Junior Member

10+ Year Member

joined:Dec 19, 2000
posts:193
votes: 0


To my knowledge, it is very hard to crawl behind such a fence, because its the server blocking. Unless of course the fence had holes in it..