homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum


 7:58 pm on Mar 5, 2009 (gmt 0)

Note sure if this was actually MSN or another org utilizing the same initials.
Perhaps even some type of MSN tool.

65.210.123.zzz - - [05/Mar/2009:07:28:44 +0000] "GET /MyFolder/MyPage.htm HTTP/1.1" 403 998 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3"
60.230.101.zz - - [05/Mar/2009:07:28:44 +0000] "GET /SameFolder/SamePage.htm HTTP/1.1" 403 998 "http://au.search.yahoo.com/search?p=widgets" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506)"



 6:08 pm on Mar 6, 2009 (gmt 0)

Was the first UA really broken, or was that a copy/paste error?


 6:32 pm on Mar 6, 2009 (gmt 0)

Search for "65.210.123":
  • Broken UAs
    "Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; ....../1.0 )"
    "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3"
  • [webmasterworld.com ]
    now my block ip list have



 9:35 pm on Mar 6, 2009 (gmt 0)

The UA's were exactly as shown.



 10:11 pm on Mar 6, 2009 (gmt 0)

Thanks. :)


 7:08 am on Mar 7, 2009 (gmt 0)

I just had some unusual accesses from this MS IP --

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; MSN 9.0;MSN 9.1;MSN 9.6; MSNbMSNI; MSNmen-us; MSNcIA)

Unusual in that they snagged home html and graphics but no JS, then requested only one part of a three-part frameset -- a split blocked if JavaScript is enabled -- and then eight more html pages, all without their unique graphics.

I can't recall ever e-mailing re access with MSN folks other than employees using their 'tide' servers so beats me who/what this was. Well, if they come back, they'll have a chance to e-mail for access:)


 8:44 am on Mar 7, 2009 (gmt 0)

These last three portions of the UA MSNbMSNI; MSNmen-us; MSNcIA and other similar variations of same, are only shown when the MS User utilizes web-access of their MSN account to view a web page.

In other words the MSN user is not using the standard offline browser on their local machine.

Whether this online version offers configuration options (or even the selective portions in your example) is unknown to me.


 6:14 pm on Mar 7, 2009 (gmt 0)

@wilderness: Does that mean it's a real person intentionally or unintentionally doing odd things? (I guess I don't know what 'web-access of their MSN account' means:)


 6:19 pm on Mar 7, 2009 (gmt 0)

Speaking of odd things -- another MSN IP:

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322)

robots.txt? YES
(But only after asking for a subdir html)

So many bots, so little time...


 6:21 pm on Mar 7, 2009 (gmt 0)

I think he means that since MSN is an ISP you can use your MSN account to access stuff on the Internet other than the web.


 6:24 pm on Mar 7, 2009 (gmt 0)

I recall a bit back one of my Moderators digging up some information that those referenced in the OP are in fact connected with MSN in some way. I'm way past my bedtime, so you'll have to let me get back to you with the specifics. I think we have some quoted stuff from the MSN folks that there is a connection to them. I'll see if she's logged on and can dig up the info after I get a few hours of downtime. Sorry to be so vague.


 7:27 pm on Mar 7, 2009 (gmt 0)

Okay, she just gave me this:

From what I remember they are a part of Microsoft but are from specific depts like Media, Entertainment that sort of thing. I have not seen any info that suggested they are a problem. They just seem to be operating separately so they can keep track of one specific area and be able to provide users with more information on that one subject.

And my log from those days has all of them like this:

Guest IP: 65.55.107 or 109 or 110.000 Whois
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322)

I see from one post up there that this hasn't changed:


Until we put a stop to it we were getting bunches of those busy little fellas/gals.

Don't know if that'll help anyone, but ...


 8:57 pm on Mar 7, 2009 (gmt 0)

@wilderness: Does that mean it's a real person intentionally or unintentionally doing odd things? (I guess I don't know what 'web-access of their MSN account' means

Somewhere there's another thread in which I inquired regarding these UA's.
I had a portions of my limited white-listing that was denying them access and it was quite difficult to make a determination of precisely what line what was the cause.

I've widget users who are constantly on the road (AOL; aside below) these users need local dial-ups across North America.
(It's similar to users that only use their YAHOO email portal to access the web)

In some instances they use their own laptop in their traveled location and in other instances they may use another's machine.
When they access their MSN account the UA variations are added and are different from when they are home-based. . . and when they access their account without using the MSN web-access-portal (i. e., IE and OE a with direct ISP-POP configurations).

Please forgive me as I'm not aware of terminology for the difference in this access.

Here are five of the added UA's that I accumulated in April 2008 (I've since since more variations):

MSN Optimized;US; MSN Optimized;US
MSN 9.0;MSN 9.1; MSNbMSNI; MSNmen-us; MSNcIA)"
MSN 9.0;MSN 9.1; MSNbMSNI; MSNmen-us; MSNcOTH)"
MSN 9.0;MSN 9.1; MSNbQ002; MSNmen-us; MSNcIA)"
MSN 9.0;MSN 9.1; MSNbVZ02; MSNmen-us; MSNcOTH)"

AOL aside:
I've many widget users that have kept AOL accounts for years because of the traveling access of dial-ups.
This past year, I'd simply has enough of AOL (when visitors were refreshing pages 10 or 20 times in succession within second or AOL was caching items (pages and images) that I did not desired cached and place restrictions on AOL visitors.



 10:42 pm on Mar 7, 2009 (gmt 0)

@wilderness: Thank you for taking the time to explain! I've long seen the gobbledygooky UA variations, of course. Even wondered if there was some commonality way back [webmasterworld.com], when 'visitors' using similar UAs only snagged thousands of favicons every week.

In the first instance I reported above, the browsing pattern was not similar to that of a real person, so regardless of their UA and the reason for an IP-only access point/portal, 65.54.154.nn, is maybe-suspect.

The second instance, with someone/thing clearly botrunning, makes 65.55.232.nn definitely suspect.


 10:39 am on Mar 8, 2009 (gmt 0)

This is another thread when I asked about one of the 65.55 ip range mentioned here

Doing a dual trip ip<->rdns brings up no host so on one server I block access. I checked on some other servers with heavy traffic and all the requests from that ip range belong to bots. I also checked for the UAs Don mentioned, (like MSNmen-us) lots of these requests are people as far I can see they come from all over the place. Few of the requests with these UAs had a bot behavior, so it's not something I rely on to block/allow access.

Even the ip/dns verification has failed in several cases where real people were behind incorrectly configured servers but they did place legit orders.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved