Someone at MS just got banned!

Was Bill Gates Surfing My site?

[carfac]

Hi:

Just saw this guy, fell into a spider trap:

131.107.137.47 - - [11/Apr/2003:01:31:08 -0600] "GET /a/deep/link.html HTTP/1.1" 200 12589 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705)"

No referer, came in on a deep link (like from a SE), and d/l pages but no images. After about 5 hits, he tried to grab a trap, and got banned. Grabbed a page every 5 secs or so...

IP resolves to Redmond.... did Bill just get himself banned?

dave

[jim_w]

http ://www.clearwaterbeachcam.com/d--skinner/spiders.html

This is ironic, when I go there I end up at…
[search.msn.com...]

(hehehehehehe)

#MSN

Does that mean the IP’s belong to msn.com and not microsoft.com? If that’s the case, I just banned all msn users. Oops. If some of that block is msn.com and not microsoft.com, that would explain a lot of this.

Does anyone know what msn.com IP’s are? Is there anyway of getting the IP block for anydomain.com?

#tide119.microsoft.com
#208.147.66.139

I get - Cable & Wireless

while it could be a new bot, that does not necessarily mean that it is a SE bot.

I should have said ‘THE SE BOT’.

[wilderness]

Jim
If you go to " georgegg " page and use any one of those
(not sure what their called?)tide28.microsoft.com

You will see that they are still active (at least registered with MS.) whether that is Microsoft or MSN IMO, is really irrelavant.

I've removed the denies from 131.107. with "egg on my face"
having gone through Arin-Whois on all those ranges I'm in the process of allowing some of those MS IP ranges back into (from denied) to my FarEast blocks.

Most everybody realizes how over-bearing I am in these matters and I believe this should resolve this issue.
Although as Pendanticist points out, there still exists the possibility of it being a sppof'd range in our logs.
Due to the recent PERSISTENT activity (131.107) and the related ranges, I'm going to accept that chance.
Hopefully in the process I won't end up with even more egg on my face ;)

Don

<BTW Jim, that page comes right up for as soon as I omit the blank space I purposely left in the URL so the link would be broken>

[jim_w]

Most everybody realizes how over-bearing I am in these matters

Yea, so am I. When you do a search for my keywords you see Motorola, GE, Honeywell, and I have had so many spy bots, that I get a little trigger happy sometimes. Banned a customer once even. (GRIN)

Pendanticist was right then. Complain to abuse@microsoft and [e-mail] and let them figure it out?

<BTW Jim, that page comes right up for as soon as I omit the blank space I purposely left in the URL so the link would be broken>

Yea, it was an attempt @ humor. Obviously a poor attempt! But once I hit Mr. Button, it was toooooooo late. :-))

Doesn’t msn do dynamic IP’s? If they do, then wouldn’t that make 131.107.137.47 microsoft.com because it was consistent? And I still have a problem with the ‘+’ thing that showed up.

[wilderness]

<snip>Complain to abuse@microsoft</snip>

I stopped emailing IP's and backones some time ago. Generally your only response is automated. In the event you find somebody lucky enough to email with? They are not aware of any web log pattern nor, do they have the ability to comapre those patterns to their User Agreeements.
Their only concern is bandwith.

I'm not all the keen on the variations in UA's either. However just denying a visitor access because of UA with out comparing that to IP is TOOOO overbearing. IMO anyway.

These logs, like the internet are an always changing thing and though we are required perception? We should also remain open-minded. Hopefully creating a worthwhile balance of both which benefits both our websites and our visitors.
<off the soap box> ;)

Don

[pendanticist]

I'm on the phone with MS at this very moment and it appears as though Mr. Birney is indeed an employee of theirs.

Be right back....

Pendanticist.

[pendanticist]

Ok. I've spoken to a receptionist at tech and she is going to determine the legitimacy of this bot, once and for all.

She asked for my phone number and I gave her my ISP addy, so I do expect to hear from her.

I will let you know as soon as I hear anything at all.

(Thanks! to NeoTrace)

Pendanticist.

[GoogleGuy]

Interesting saga--the hotmail address is a little strange. Keep us posted, pendanticist. :)

[jns594]

All our sites got spidered yesterday by the same bot. Here is a sample of the IIS log file:

2003-04-24 20:08:56 131.107.163.50 - myserverip 80 GET /robots.txt - 404 MicrosoftPrototypeCrawler+(How's+my+crawling?+mailto:[e-mail]) - -

2003-04-24 20:08:57 131.107.163.50 - myserverip 80 GET /Default.asp - 200 MicrosoftPrototypeCrawler+(How's+my+crawling?+mailto:[e-mail]) - -

2003-04-24 20:09:24 131.107.163.50 - myserverip 80 GET /robots.txt - 404 MicrosoftPrototypeCrawler+(How's+my+crawling?+mailto:[e-mail]) - -

2003-04-24 20:09:24 131.107.163.50 - myserverip 80 GET /whatsnew.asp - 200 MicrosoftPrototypeCrawler+(How's+my+crawling?+mailto:[e-mail]) - -

[kwngian]

I got hit too by this spider.

Is it a Microsoft loyalty probe?

Anyone running anything other than IIS?

[jim_w]

Yea, I have a sun unix box.