MSN IP's?

[Brett_Tabke]

Showing all these with a msnbot agent. All these actual msn bots?

65.54.165.37
65.54.165.62
65.54.165.38

There is one, I am amost certain is a human.

[wilderness]

Brett,
My MSN's for today:

65.55.213.38 (had the media bot also attempting a mass crawl before the month's end from the same IP)
65.55.209.250
207.46.98.112

[volatilegx]

I can verify all of the IPs wilderness posted as being MSN spiders I've seen before, but Brett, I haven't seen those IPs you posted crawling. Do you have their complete user agents?

The reverse DNS lookups come up with a bad PTR A record:

65.54.165.37 PTR record: by1sch4030210.phx.gbl. [TTL 3600s] [A=None] *ERROR* There is no A record for by1sch4030210.phx.gbl. (may be negatively cached).

by1sch4030210.phx.gbl as a hostname on the internet, does not exist. Maybe it's some kind of intranet hostname?

I've seen that sort of thing before with MSN bots.

[bobothecat]

65.54.165.37
65.54.165.62
65.54.165.38

Yes, they are Microsoft IP's... now whether their bots or not... who knows with them, I see strange things from them all of the time.

OrgName: Microsoft Corp
OrgID: MSFT
Address: One Microsoft Way
City: Redmond
StateProv: WA
PostalCode: 98052
Country: US

NetRange: 65.52.0.0 - 65.55.255.255

[incrediBILL]

OK, here's a cluster of MSNBOT hits that keep coming from China and the whois links some to
MICROSOFT-CO Microsft (China) and others to CHINATELECOM-BJ.

The list of IPs below are shown in consecutive order of how they accessed the server within a few minutes time so I believe they are all really from Microsoft.

202.96.51.153 "msnbot/1.0 (+http://search.msn.com/msnbot.htm)"
202.96.51.158 "msnbot/1.0 (+http://search.msn.com/msnbot.htm)"
219.142.53.28 "msnbot/1.0 (+http://search.msn.com/msnbot.htm)"
219.142.53.25 "msnbot/1.0 (+http://search.msn.com/msnbot.htm)"
219.142.53.31 "msnbot/1.0 (+http://search.msn.com/msnbot.htm)"
202.96.51.152 "msnbot/1.0 (+http://search.msn.com/msnbot.htm)"
202.96.51.154 "msnbot/1.0 (+http://search.msn.com/msnbot.htm)"
219.142.53.26 "msnbot/1.0 (+http://search.msn.com/msnbot.htm)"
219.142.53.22 "msnbot/1.0 (+http://search.msn.com/msnbot.htm)"
219.142.53.19 "msnbot/1.0 (+http://search.msn.com/msnbot.htm)"

Now the big question is WHY?

Bandwidth cheaper in China?

Should I let them in or continue to just let them bounce off my site?

[wilderness]

Bill,
Over the Christmas Holidays I watched a replay of a Charlie Rose show that was filed at Stanford.
It was all computer geeks and the topic was Green Peace and global warming.

I don't remember the numbers of how much energy it took to run a data center in a major city, it was very high and extreme.
The general consensus was that (besides already takking place) in the future the large data centers would be located in cooler climates that offer less expensive budgets and in more remote areas.

[Pfui]

Howdy, all. FWIW --

(1.) Re Brett's maybe-bot numbers: Just unnecessary double-hits to the same favicon. Hmm...

65.54.154.16 - - [01/Mar/2007:00:58:14...] /favicon.ico HTTP/1.0" 200 2238 "-"
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322;
.NET CLR 2.0.50727; MSN 9.0;MSN 9.1; MSNbMSNI; MSNmen-us; MSNcIA)"
(same UA each time)
65.54.154.16 - - [01/Mar/2007:00:58:19...] /favicon.ico
65.54.155.52 - - [05/Mar/2007:05:22:50...] /favicon.ico
65.54.155.52 - - [05/Mar/2007:05:22:53...] /favicon.ico

(2.) Re MS's maybe-bots from China: I block/redirect them to a private server, to a page with a tracking JPG. They follow, and skip the JPG. More importantly, they also skip the robots.txt...

219.142.53.27 - - [01/Mar/2007:15:52:36...] /403.html
"msnbot/1.0 (+http://search.msn.com/msnbot.htm)"
(same UA each time)
219.142.53.21 - - [01/Mar/2007:21:00:56...] /403.html
219.142.53.25 - - [02/Mar/2007:20:42:39...] /403.html
219.142.53.16 - - [03/Mar/2007:08:48:29...] /403.html
219.142.53.28 - - [03/Mar/2007:12:22:24...] /403.html
219.142.53.22 - - [04/Mar/2007:04:13:35...] /403.html
219.142.53.20 - - [05/Mar/2007:08:23:05...] /403.html

Prior: [webmasterworld.com...]

(3.) I only see msnbot and msnbot-media from legit HOSTs, not IPs, and not dovetailing with the no-HOST Chinese IPs, Thus far today (other than robots.txt omitted), spaced by UA for readability...

livebot-65-55-209-47.search.live.com - - [05/Mar/2007:08:30:46...] /robots.txt
"msnbot/1.0 (+http://search.msn.com/msnbot.htm)"

livebot-65-55-212-167.search.live.com - - [05/Mar/2007:09:21:55...] /robots.txt
"msnbot-media/1.0 (+http://search.msn.com/msnbot.htm)"
livebot-65-55-235-162.search.live.com - - [05/Mar/2007:11:34:08...] /robots.txt
"msnbot-media/1.0 (+http://search.msn.com/msnbot.htm)"

livebot-65-55-209-47.search.live.com - - [05/Mar/2007:11:48:42...] /robots.txt
"msnbot/1.0 (+http://search.msn.com/msnbot.htm)"

by1sch4030314.phx.gbl - - [05/Mar/2007:11:59:31...] /robots.txt
"msnbot/1.0 (+http://search.msn.com/msnbot.htm)"

Okay. Back to work. :)

[volatilegx]

Excellent; thanks Pfui