Welcome to WebmasterWorld Guest from 54.161.25.142

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

MIIxpc?

     
9:32 pm on Mar 13, 2001 (gmt 0)

10+ Year Member



I don't currently block anyone. However after blowing my bandwidth limit last month I intend to start doing so now.

Does anyone know why I shouldn't add these 3 to the list ?

JennyBot
MIIxpc
teoma_agent3

10:09 pm on Mar 13, 2001 (gmt 0)

10+ Year Member



User-agent: Googlebot
Disallow: /
10:34 pm on Mar 13, 2001 (gmt 0)

WebmasterWorld Senior Member mivox is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Why would you want to ban Google from your entire site?

Air

10:01 pm on Mar 15, 2001 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>MIIxpc

Not absolutely sure, but there is some indication that this may be altavista.nl or altavista.de, can anyone confirm?

3:51 pm on Mar 16, 2001 (gmt 0)

10+ Year Member



Air,
I notice that spider, too. I donīt know where it comes from, but I would guess it spiders the livesearches of another engines.
11:26 pm on Mar 27, 2001 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Isn't this for the robots.txt file? Am I missing something? Most of these robots would never check the robots.txt file, right?

Dan

1:42 pm on Mar 29, 2001 (gmt 0)

10+ Year Member



>MIIxpc

>I notice that spider, too. I donīt know where it comes >from, but I would guess it spiders the livesearches of >another engines.

oLeon, do you think that is what is going on here ?

195.121.6.106 - - [28/Mar/2001:06:35:55 -0500] "GET / HTTP/1.1" 200 9693 "http://195.121.7.86/cgi-bin/zoeken/avsearch.cgi?pg=q&q=border+terrier&kl=XX&what=web&stq=10" "Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt)"

195.121.6.106 - - [28/Mar/2001:06:35:59 -0500] "GET /images/film.jpg HTTP/1.1" 200 5911 "http://www.champdogs.co.uk/html/home.html" "Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt)"

212.78.177.71 - - [28/Mar/2001:06:36:00 -0500] "GET /images/film.jpg HTTP/1.0" 200 5911 "-" "MIIxpc/4.2"

195.121.6.106 - - [28/Mar/2001:06:36:12 -0500] "GET /html/search_menu.htm HTTP/1.1" 200 1815 "-" "Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt)"

195.121.6.106 - - [28/Mar/2001:06:36:12 -0500] "GET /html/search.htm HTTP/1.1" 200 824 "http://www.champdogs.co.uk/html/master_menu.htm" "Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt)"

212.78.177.71 - - [28/Mar/2001:06:36:12 -0500] "GET /html/search.htm HTTP/1.0" 200 824 "-" "MIIxpc/4.2"

212.78.177.70 - - [28/Mar/2001:06:36:12 -0500] "GET /html/search_menu.htm HTTP/1.0" 200 1815 "-" "MIIxpc/4.2"

It followed the surfer right round my site, taking the identical pages including the graphics.

10:19 pm on Apr 19, 2001 (gmt 0)

10+ Year Member



It's an accelerator of Mirrorimage.net. Just resolve the IP and then do a whois on the domain xpc-mii.net and you'll end up at mirrorimage.net.
They offer some kind of shared proxy-cache. Everytime you get visited by websurfer having enabled a proxy that is using this shared proxy-cache, you'll find this thing hitting your site some 30 minutes later...
11:59 pm on May 31, 2001 (gmt 0)

10+ Year Member



Is it supposed to crawl behind a .htaccess fence?

On my site it also follows the behaviour of another user.

12:03 am on Jun 5, 2001 (gmt 0)

10+ Year Member



To my knowledge, it is very hard to crawl behind such a fence, because its the server blocking. Unless of course the fence had holes in it..
 

Featured Threads

Hot Threads This Week

Hot Threads This Month