MSN fakes referrers

[SEOPTI]

This has been discussed in 2007:
[webmasterworld.com...]

They do it again, I see hundreds of fake visitors from MSN IPs across all of my domains.

Are there any news what they try to accomplish by doing this?

[blend27]

I am not going to be any less sarcastic than I usually am, but that looks like "Fill Out a Form" link.

There is still no response to why this is happening. hmmm, Friday word game anyone?

-- Thank you once again! MSN fakes referrers

[wilderness]

Went through 24-pages of results and no links to my sites appeared on this [url=http:// search.live.com/results.aspx?q=gauman]search phrase[/url], however a visitor entered one of my pages on this same search.

[edited by: wilderness at 4:16 pm (utc) on April 28, 2009]

[msndude]

Hi,

It may seem different, but I really do enjoy helping people overcome issues. We are still working to correct this issue.

Out of curiosity, have any of you tried setting crawl-delays in your robots.txt or set conditional gets to slow the requests?

Brett

[SEOPTI]

Yes, crawl-delays have been set, but of course a crawl delay won't sort out this issue, it simply delays the fake requests.

[incrediBILL]

Yes, I've been using crawl-delays for years now and every now and then msnbot just goes off the deep end and brought my server down 2 or 3 times last year until I temporarily blocked it altogether until it decided to behave itself.

[msndude]

I just spoke with the team. They said they are working on a fix for this, but the feature causing the problem is turned off currently, so you shouldn't be seeing any fake referrals from us.

[Samizdata]

Thanks msndude, we really appreciate the feedback.

...

[sem4u]

I still see them coming through here.

[JS_Harris]

EPN affiliates apparently noticed this again on April 19th, Interestingly enough it appears that Google Analytics didn't count the massive temporary increase in traffic but EPN reported it and 3rd party tracking software/sources confirmed the "carpet bombing".

The nofollow tag was ignored of course.

[blend27]

65.55.110.86 at 2009-04-30 00:58:51.793
65.55.109.162 at 2009-04-29 16:19:49.923
65.55.232.49 at 2009-04-29 15:43:22.970
65.55.232.49 at 2009-04-29 14:53:50.553
65.55.107.239 at 2009-04-29 06:11:25.913
65.55.109.19 at 2009-04-28 23:52:58.977
65.55.109.12 at 2009-04-28 22:43:18.100
65.55.110.107 at 2009-04-28 19:29:28.467
65.55.109.234 at 2009-04-28 19:24:47.077
65.55.110.122 at 2009-04-28 19:15:08.140
65.55.109.196 at 2009-04-28 17:04:53.120
65.55.109.118 at 2009-04-28 14:02:28.837
65.55.110.192 at 2009-04-28 12:30:52.497
65.55.110.237 at 2009-04-28 10:34:44.017
65.55.109.210 at 2009-04-28 09:23:40.790
65.55.109.211 at 2009-04-28 08:29:07.243
65.55.107.233 at 2009-04-28 06:11:22.050

NetRange: 65.52.0.0 - 65.55.255.255
CIDR: 65.52.0.0/14
NetName: MICROSOFT-1BLK
NetHandle: NET-65-52-0-0-1
NetType: Direct Assignment

They are still comming, so

[nealrodriguez]

Is there really a reason to hit a site with fake search requests for "phone" or "address" hundreds of times a day?

chronic amnesia?

[antoineFR]

same with french web site. Fake referer from Microsoft IP.

But also fake agent, and even javascript simulation to get cookies !

Ex today on a lot of sites : IP 65.55.110.242

Agent : Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322)
Referer : [search.live.com...]

If i was not sure at 100% that microsoft is an honnest company i would say that it is done to fake world stats about Live use...

Antoine.

[confuscius]

As the original poster back in 2007, I am pleased to see that action now seems to be forthcoming on this issue. I took the decision to ban MSN from all of my domains except one that I left open - this has been happening continuously during the period and once MSN finds you then there is no escape.

Given past responses from MSN then I am not getting my hopes up that this will be fixed but look forward to being surprised.

Is there a 'Guiness World Record for the slowest technical fix of all time'? - if so, then it is about time MSN received the accolade they deserve.

[particleman]

Glad this has come up, I've noticed this for a long time in our stats. Keywords seem to be centered around the subject of the site also. Can someone actually explain what this is about?

[koan]

This was a serious pet peeves of mine with Live, glad to hear it's finally getting some attention. None too soon!

[msndude]

Thanks for keeping me updated. I'll take this back to the team. I'll update this thread as soon as I have more info.

If any of you have recent screen caps that you wouldn't mind sharing, could you please send them to me at [e-mail] with your domain name and this thread's title in the subject line?

Thanks,

Brett

[msndude]

Writing a new blog post regarding these issues. Should be up on our webmaster blog in a couple days/ week max.

Brett

[Pfui]

While we await Brett's blog post (thank you), I thought I'd mention that MSN's generic-word, fake-referer scheme/program is baaaaack, or never left.

Hitting html, js, and css files, the following are from two disparate sites today, and earlier this month. Examples chosen because of the IP/Host mix.

65.55.110.208
[search.live.com...]

65.55.110.42
[search.live.com...]

msnbot-65-55-109-241.search.msn.com
[search.live.com...]

The UA is always: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322)

[Pfui]

FWIW, MSN is still faking it. From a quick skim of my logs (vs grepping for all instances):

06-03: 65.55.109.64
06-04: msnbot-65-55-110-207.search.msn.com
06-05: 65.55.110.110

REF x3: [search.live.com...]
UA x3: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322)
URIs: Three different html files.

I deny MSN's usual fakes (e.g., SetEnvIfNoCase Referer "q=secrets" scram) but doing so doesn't deter.

[Pfui]

And one more, just in, because this fake q= is nothing but a waste of everyone's resources:

msnbot-65-55-109-147.search.msn.com
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322)
[search.live.com...]

(Gimme a break.)

[TravelMan]

I'm getting this too - I'm tired of these jokers, msndude, you bing guys are doing yourself noooo favours. Can someone just turn this nonsense off!

[incrediBILL]

Writing a new blog post regarding these issues. Should be up on our webmaster blog in a couple days/ week max.

Thanks MSNDUDE, we'll be looking forward to it.

C'mon guys, give MSNDUDE some respect, he's here trying to help and explain what's going on with this.

[BillyS]

>>FWIW, MSN is still faking it

Of course they are, this is one of the ways they detect spam. They're not going to fix this anytime soon.

I've already started blocking them...

[blend27]

I think any half baked blackhat out there knows that THIS BOT comes from CIDR: 65.52.0.0/14, how easy is it to cloak the JS/CSS content to that if they wanted to?, think about it.

.classI{text-indent: <cfif not request.bot>-999</cfif>4px;} :)

JQuery UI Lib is 300 kb minified, it was pull down by this BOT more than several times a day, NO MORE - 403 with no changes in rankings so far.

Patiently waiting for that blog post by MSNDUDE...

Blend27

[rytis]

MS just searched my site for "windows", I am willing to help, but how?

[blend27]

and my site for "company", musta been a lonely Bot looking for something to Bing. Too bad, I am already hitched... :)

[BillyS]

I just saw another fake one from Live... it came from 65.55.110.186.

[edited by: BillyS at 1:54 am (utc) on June 17, 2009]

[Pfui]

MSN's fake-rate is higher than they ever, with multiples and even dupes, too. All ridiculously broad and resource-wasting:

06/18 13:12:59 [search.live.com...]
06/18 13:13:52 [search.live.com...]
06/18 13:13:53 [search.live.com...]
06/18 14:10:04 [search.live.com...]

All of those came from this real Host:

msnbot-65-55-104-nn.search.msn.com
(nn = multiple numbers)

All Hosts used this cloaked UA --

Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4325; .NET CLR 2.0.40607; .NET CLR 3.0.04506.648)

All 403'd. C'est la vie.

[dstiles]

Has anyone seen a real msn referral querystring that's as simple as these? Would ^q=[a-z]$ ever trap a real "live" (or bing) referral?

I'm in the process of setting up a new stats analysis system for my clients and have had to warn them that MS referrals are artificially high and are probably nearer 0.3% than the 3.3% that seems typical in the logs.

[GaryK]

Now that it's happening to me I have a better handle on what this thread is all about.

06/14/2009:
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4325; .NET CLR 2.0.40607; .NET CLR 3.0.04506.648)
65.55.104.67
msnbot-65-55-104-67.search.msn.com
[search.live.com...]

06/16/2009:
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648)
65.55.104.65
msnbot-65-55-104-65.search.msn.com
[search.live.com...]

06/17/2009:
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SLCC1; .NET CLR 1.1.4325; .NET CLR 2.0.40607; .NET CLR 3.0.30729; .NET CLR 3.5.30707; MS-RTC LM 8)
65.55.104.61
msnbot-65-55-104-61.search.msn.com
[search.live.com...]

06/18/2009:
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SLCC1; .NET CLR 1.1.4322; .NET CLR 2.0.40607; .NET CLR 3.0.04506.648)
65.55.104.62
msnbot-65-55-104-62.search.msn.com
[search.live.com...]

06/20/2009:
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SLCC1; .NET CLR 1.1.4325; .NET CLR 2.0.40607; .NET CLR 3.0.30729; .NET CLR 3.5.30707; InfoPath.2)
65.55.104.72
msnbot-65-55-104-72.search.msn.com
[search.live.com...]