homepage Welcome to WebmasterWorld Guest from 54.145.209.77
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

This 131 message thread spans 5 pages: < < 131 ( 1 [2] 3 4 5 > >     
MSN fakes referrers
SEOPTI

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3875363 posted 6:14 pm on Mar 20, 2009 (gmt 0)

This has been discussed in 2007:
[webmasterworld.com...]

They do it again, I see hundreds of fake visitors from MSN IPs across all of my domains.

Are there any news what they try to accomplish by doing this?

 

blend27

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3875363 posted 9:35 am on Apr 28, 2009 (gmt 0)

I am not going to be any less sarcastic than I usually am, but that looks like "Fill Out a Form" link.

There is still no response to why this is happening. hmmm, Friday word game anyone?

-- Thank you once again! MSN fakes referrers

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 3875363 posted 4:13 pm on Apr 28, 2009 (gmt 0)

Went through 24-pages of results and no links to my sites appeared on this [url=http:// search.live.com/results.aspx?q=gauman]search phrase[/url], however a visitor entered one of my pages on this same search.

[edited by: wilderness at 4:16 pm (utc) on April 28, 2009]

msndude

10+ Year Member



 
Msg#: 3875363 posted 4:13 pm on Apr 28, 2009 (gmt 0)

Hi,

It may seem different, but I really do enjoy helping people overcome issues. We are still working to correct this issue.

Out of curiosity, have any of you tried setting crawl-delays in your robots.txt or set conditional gets to slow the requests?

Brett

SEOPTI

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3875363 posted 5:50 pm on Apr 28, 2009 (gmt 0)

Yes, crawl-delays have been set, but of course a crawl delay won't sort out this issue, it simply delays the fake requests.

incrediBILL

WebmasterWorld Administrator incredibill us a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month



 
Msg#: 3875363 posted 6:09 pm on Apr 28, 2009 (gmt 0)

Yes, I've been using crawl-delays for years now and every now and then msnbot just goes off the deep end and brought my server down 2 or 3 times last year until I temporarily blocked it altogether until it decided to behave itself.

msndude

10+ Year Member



 
Msg#: 3875363 posted 7:15 pm on Apr 28, 2009 (gmt 0)

I just spoke with the team. They said they are working on a fix for this, but the feature causing the problem is turned off currently, so you shouldn't be seeing any fake referrals from us.

Samizdata

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3875363 posted 7:17 pm on Apr 28, 2009 (gmt 0)

Thanks msndude, we really appreciate the feedback.

...

sem4u

WebmasterWorld Senior Member sem4u us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3875363 posted 8:22 am on Apr 30, 2009 (gmt 0)

I still see them coming through here.

JS_Harris

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3875363 posted 11:44 am on Apr 30, 2009 (gmt 0)

EPN affiliates apparently noticed this again on April 19th, Interestingly enough it appears that Google Analytics didn't count the massive temporary increase in traffic but EPN reported it and 3rd party tracking software/sources confirmed the "carpet bombing".

The nofollow tag was ignored of course.

blend27

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3875363 posted 2:26 pm on Apr 30, 2009 (gmt 0)

65.55.110.86 at 2009-04-30 00:58:51.793
65.55.109.162 at 2009-04-29 16:19:49.923
65.55.232.49 at 2009-04-29 15:43:22.970
65.55.232.49 at 2009-04-29 14:53:50.553
65.55.107.239 at 2009-04-29 06:11:25.913
65.55.109.19 at 2009-04-28 23:52:58.977
65.55.109.12 at 2009-04-28 22:43:18.100
65.55.110.107 at 2009-04-28 19:29:28.467
65.55.109.234 at 2009-04-28 19:24:47.077
65.55.110.122 at 2009-04-28 19:15:08.140
65.55.109.196 at 2009-04-28 17:04:53.120
65.55.109.118 at 2009-04-28 14:02:28.837
65.55.110.192 at 2009-04-28 12:30:52.497
65.55.110.237 at 2009-04-28 10:34:44.017
65.55.109.210 at 2009-04-28 09:23:40.790
65.55.109.211 at 2009-04-28 08:29:07.243
65.55.107.233 at 2009-04-28 06:11:22.050

NetRange: 65.52.0.0 - 65.55.255.255
CIDR: 65.52.0.0/14
NetName: MICROSOFT-1BLK
NetHandle: NET-65-52-0-0-1
NetType: Direct Assignment

They are still comming, so

nealrodriguez

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3875363 posted 6:19 pm on Apr 30, 2009 (gmt 0)

Is there really a reason to hit a site with fake search requests for "phone" or "address" hundreds of times a day?

chronic amnesia?

antoineFR

5+ Year Member



 
Msg#: 3875363 posted 6:50 pm on Apr 30, 2009 (gmt 0)

same with french web site. Fake referer from Microsoft IP.

But also fake agent, and even javascript simulation to get cookies !

Ex today on a lot of sites : IP 65.55.110.242

Agent : Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322)
Referer : [search.live.com...]

If i was not sure at 100% that microsoft is an honnest company i would say that it is done to fake world stats about Live use...

Antoine.

confuscius

5+ Year Member



 
Msg#: 3875363 posted 6:54 pm on Apr 30, 2009 (gmt 0)

As the original poster back in 2007, I am pleased to see that action now seems to be forthcoming on this issue. I took the decision to ban MSN from all of my domains except one that I left open - this has been happening continuously during the period and once MSN finds you then there is no escape.

Given past responses from MSN then I am not getting my hopes up that this will be fixed but look forward to being surprised.

Is there a 'Guiness World Record for the slowest technical fix of all time'? - if so, then it is about time MSN received the accolade they deserve.

particleman

5+ Year Member



 
Msg#: 3875363 posted 11:12 pm on Apr 30, 2009 (gmt 0)

Glad this has come up, I've noticed this for a long time in our stats. Keywords seem to be centered around the subject of the site also. Can someone actually explain what this is about?

koan

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3875363 posted 11:16 pm on May 1, 2009 (gmt 0)

This was a serious pet peeves of mine with Live, glad to hear it's finally getting some attention. None too soon!

msndude

10+ Year Member



 
Msg#: 3875363 posted 4:06 am on May 2, 2009 (gmt 0)

Thanks for keeping me updated. I'll take this back to the team. I'll update this thread as soon as I have more info.

If any of you have recent screen caps that you wouldn't mind sharing, could you please send them to me at lswmc@microsoft.com with your domain name and this thread's title in the subject line?

Thanks,

Brett

msndude

10+ Year Member



 
Msg#: 3875363 posted 10:20 pm on May 18, 2009 (gmt 0)

Writing a new blog post regarding these issues. Should be up on our webmaster blog in a couple days/ week max.

Brett

Pfui

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3875363 posted 5:35 am on May 23, 2009 (gmt 0)

While we await Brett's blog post (thank you), I thought I'd mention that MSN's generic-word, fake-referer scheme/program is baaaaack, or never left.

Hitting html, js, and css files, the following are from two disparate sites today, and earlier this month. Examples chosen because of the IP/Host mix.

65.55.110.208
[search.live.com...]

65.55.110.42
[search.live.com...]

msnbot-65-55-109-241.search.msn.com
[search.live.com...]

The UA is always: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322)

Pfui

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3875363 posted 7:30 pm on Jun 5, 2009 (gmt 0)

FWIW, MSN is still faking it. From a quick skim of my logs (vs grepping for all instances):

06-03: 65.55.109.64
06-04: msnbot-65-55-110-207.search.msn.com
06-05: 65.55.110.110

REF x3: [search.live.com...]
UA x3: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322)
URIs: Three different html files.

I deny MSN's usual fakes (e.g., SetEnvIfNoCase Referer "q=secrets" scram) but doing so doesn't deter.

Pfui

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3875363 posted 8:56 pm on Jun 5, 2009 (gmt 0)

And one more, just in, because this fake q= is nothing but a waste of everyone's resources:

msnbot-65-55-109-147.search.msn.com
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322)
[search.live.com...]

(Gimme a break.)

TravelMan

10+ Year Member



 
Msg#: 3875363 posted 9:25 am on Jun 6, 2009 (gmt 0)

I'm getting this too - I'm tired of these jokers, msndude, you bing guys are doing yourself noooo favours. Can someone just turn this nonsense off!

incrediBILL

WebmasterWorld Administrator incredibill us a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month



 
Msg#: 3875363 posted 10:29 pm on Jun 6, 2009 (gmt 0)

Writing a new blog post regarding these issues. Should be up on our webmaster blog in a couple days/ week max.

Thanks MSNDUDE, we'll be looking forward to it.

C'mon guys, give MSNDUDE some respect, he's here trying to help and explain what's going on with this.

BillyS

WebmasterWorld Senior Member billys us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3875363 posted 11:33 pm on Jun 6, 2009 (gmt 0)

>>FWIW, MSN is still faking it

Of course they are, this is one of the ways they detect spam. They're not going to fix this anytime soon.

I've already started blocking them...

blend27

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3875363 posted 2:57 pm on Jun 7, 2009 (gmt 0)

I think any half baked blackhat out there knows that THIS BOT comes from CIDR: 65.52.0.0/14, how easy is it to cloak the JS/CSS content to that if they wanted to?, think about it.

.classI{text-indent: <cfif not request.bot>-999</cfif>4px;} :)

JQuery UI Lib is 300 kb minified, it was pull down by this BOT more than several times a day, NO MORE - 403 with no changes in rankings so far.

Patiently waiting for that blog post by MSNDUDE...

Blend27

rytis

10+ Year Member



 
Msg#: 3875363 posted 11:54 am on Jun 16, 2009 (gmt 0)

MS just searched my site for "windows", I am willing to help, but how?

blend27

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3875363 posted 5:53 pm on Jun 16, 2009 (gmt 0)

and my site for "company", musta been a lonely Bot looking for something to Bing. Too bad, I am already hitched... :)

BillyS

WebmasterWorld Senior Member billys us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3875363 posted 1:54 am on Jun 17, 2009 (gmt 0)

I just saw another fake one from Live... it came from 65.55.110.186.

[edited by: BillyS at 1:54 am (utc) on June 17, 2009]

Pfui

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3875363 posted 5:23 pm on Jun 20, 2009 (gmt 0)

MSN's fake-rate is higher than they ever, with multiples and even dupes, too. All ridiculously broad and resource-wasting:

06/18 13:12:59 [search.live.com...]
06/18 13:13:52 [search.live.com...]
06/18 13:13:53 [search.live.com...]
06/18 14:10:04 [search.live.com...]

All of those came from this real Host:

msnbot-65-55-104-nn.search.msn.com
(nn = multiple numbers)

All Hosts used this cloaked UA --

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4325; .NET CLR 2.0.40607; .NET CLR 3.0.04506.648)

All 403'd. C'est la vie.

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 3875363 posted 8:25 pm on Jun 20, 2009 (gmt 0)

Has anyone seen a real msn referral querystring that's as simple as these? Would ^q=[a-z]$ ever trap a real "live" (or bing) referral?

I'm in the process of setting up a new stats analysis system for my clients and have had to warn them that MS referrals are artificially high and are probably nearer 0.3% than the 3.3% that seems typical in the logs.

GaryK

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3875363 posted 11:34 pm on Jun 21, 2009 (gmt 0)

Now that it's happening to me I have a better handle on what this thread is all about.

06/14/2009:
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4325; .NET CLR 2.0.40607; .NET CLR 3.0.04506.648)
65.55.104.67
msnbot-65-55-104-67.search.msn.com
[search.live.com...]

06/16/2009:
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648)
65.55.104.65
msnbot-65-55-104-65.search.msn.com
[search.live.com...]

06/17/2009:
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SLCC1; .NET CLR 1.1.4325; .NET CLR 2.0.40607; .NET CLR 3.0.30729; .NET CLR 3.5.30707; MS-RTC LM 8)
65.55.104.61
msnbot-65-55-104-61.search.msn.com
[search.live.com...]

06/18/2009:
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SLCC1; .NET CLR 1.1.4322; .NET CLR 2.0.40607; .NET CLR 3.0.04506.648)
65.55.104.62
msnbot-65-55-104-62.search.msn.com
[search.live.com...]

06/20/2009:
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SLCC1; .NET CLR 1.1.4325; .NET CLR 2.0.40607; .NET CLR 3.0.30729; .NET CLR 3.5.30707; InfoPath.2)
65.55.104.72
msnbot-65-55-104-72.search.msn.com
[search.live.com...]

GaryK

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3875363 posted 5:45 pm on Jun 28, 2009 (gmt 0)

Another week and yet more fake referrers from MSN. Personally I've had enough. I just don't get enough traffic from MSN to make this worthwhile so as of now they're banned from all my sites.

6/24/2009
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SLCC1; .NET CLR 1.1.4325; .NET CLR 2.0.50727; .NET CLR 3.0.30729; .NET CLR 3.5.30707; MS-RTC LM 8)
65.55.104.64
msnbot-65-55-104-64.search.msn.com
[search.live.com...]

6/27/2009
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SLCC1; .NET CLR 1.1.4325; .NET CLR 2.0.50727; .NET CLR 3.0.30729; .NET CLR 3.5.30729; MS-RTC LM 8)
65.55.104.63
msnbot-65-55-104-63.search.msn.com
[search.live.com...]

This 131 message thread spans 5 pages: < < 131 ( 1 [2] 3 4 5 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved