MSN fakes referrers

[SEOPTI]

This has been discussed in 2007:
[webmasterworld.com...]

They do it again, I see hundreds of fake visitors from MSN IPs across all of my domains.

Are there any news what they try to accomplish by doing this?

[wilderness]

SEOPTI,
There's been multiple discussions of the "search.live" refers in numerous MSN threads.

I don't recall a valid explanation being presented, however I may be mistaken.

In October of last year, I had partial key word phrases, which resulted in multiple page request in succession.
EX:
Valid phrase= "widget house".
Search1= widget
Search2=house

Both secondary searches were/are invalid on my sites.

I grew weary of the daily requests and denied ALL search.live refers for a time.
Eventually these request ceased and I removed the denial.

Sorry I'm unable to be more help.

Don

[dstiles]

Wilderness - did denying them affect MSN searches / results in any way? I have to admit their junk hits are annoying.

[wilderness]

did denying them affect MSN searches / results in any way?

dstiles,
No!

However in all honesty, there are more "search.live" refers these days than "msn" refers.

The majority of the "search.live" refers to my sites are on topic.

[dstiles]

Thanks. I may consider blocking them for a while in that case. MSN has never been a good deliverer anyway, despite constantly crawling the sites.

[blend27]

-- did denying them affect MSN searches --

NO, not the Arin/Ripe Traffic at least.

[SEOPTI]

I really don't know what to do with all those fake requests from MSN, they are bombing my sites with requests like "phone" or "city" .. this really screws up my statistics.

Maybe they are trying to fool people so it looks like MSN gains market share? What a cool strategy!

[edited by: SEOPTI at 4:57 pm (utc) on Mar. 25, 2009]

[wilderness]

really don't know what to do with all those fake requests from MSN, they are bombing my sites with requests like "phone" or "city" .. this really screws up my statistics.

In the event the refer terms are the same?
An option of denial based on same is possible, although not 100% effective.

Had some visitors that were bombing my sites regularly with:

"contact mail email phone fax tel"

[SEOPTI]

400 fake requests with the word "phone" today and counting for one of my sites from MSN, what a dumb machine!

[SEOPTI]

I think I have a solution, maybe MSN is trying to spot mars people on my sites? ;)

[Ocean10000]

Best that I can tell, they are trying to use these automated queries, which are all on the MSN Bot ranges, to improve their search engine results. My guess is that these queries do not go though as many filters as the production Search does. And are designed to show some indirect terms, which might not be on the page, but on pages referring to your page, or common in your nitch. They may also be designed to detect if you alter your page based on those search terms entered.

I know myself I just filter these out of my resulting Search terms and Referrer stats based on ip range. I ignore all search terms and referrers from known search engine bot ranges, so they do not skew my stats.

[MS_Jason]

SEOPTI,

If you keep having the problem, please contact Betsy Aoki. You can get hold of her via the contact form on her blog: [blogs.msdn.com...]

She can make sure your issue gets to the right person on the search team.

Cheers,
Jason

[incrediBILL]

Jason, Welcome to WebmasterWorld, and thanks for the contact because many of us have similar bizarre issues with MSNBOT and other strange things crawling out of MS and any help to stop this or explain it is always greatly appreciated.

[Umbra]

I didn't understand MS_Jason's reply. So many of us have seen this strange activity -- are we all supposed to separately contact Betsy, thus requiring her to individually respond to all of us? Isn't it more practical if MS_Jason took advantage of this public forum to explain what was going on?

Or is he implying that the strange activity should now have ceased and we should only contact Betsy if we're still seeing it?

[edited by: Umbra at 3:26 pm (utc) on April 21, 2009]

[Frank_Rizzo]

I had a similar issue with the msnbot - crawling non existent pages.

Contacted microsoft live support who said they would escalate it. That was over a week ago. Nothing heard since.

Betsy is going to be busy.

[incrediBILL]

One thing I've noticed are 3rd party sites deliberately linking to pages that don't exist in what it looks like they are doing is trying to make MSNBOT (and others) unwittingly hack sites.

For instance, one time I found msnbot shoving some 7K long URIs into my server for some reason which caused the server to hiccup, hopefully that's all it did.

All of these requested URIs had a huge binary string embedded that typically starts with "%C3%83%C6%92%C3%86%E2%80%99%C3%83%E2%80" and continues.

The requests look like this:

example.com/my-page-%C3%83%C6%92%C3%86%E2%80%99%C3%83%E2%80....-here.html

I did a search in Live and sure enough, there appear to be sites out there indexed that are embedding these strings in URIs either for some purpose and some of those sites are linked to malware, so draw your own conclusion.

[msndude]

Hi,

This is Brett Yount from Live Search's Webmaster Center. I would like to apologize for the inconvenience this is causing. We are working to correct this issue ASAP.

[blend27]

Brett,

Thanks for stopping by.

*ttp://search.live.com/results.aspx?q=string

Yes, the actual keyword is a word "STRING". Why are your bots wasting my bandwidth, server resources and most precious - My Time?

We constantly SCORE about useless Scrapers and Spammers for wasting our time.

[webmasterworld.com...]

Why are your bots contributing to wasted energy?

[GaryK]

Might I respectfully suggest we give Brett a chance to get this issue resolved? Then if they are still wasting our bandwidth we can gripe some more. We've griped. Gotten the attention we want. Now it's time to be patient for just a bit. :)

[SEOPTI]

Thank you MS_Jason and Brett for looking into it.

[sem4u]

This has been affecting many of my sites. It is good to see the issue is being looked into.

[Rosalind]

I'm also seeing an increase in fake referrer activity from MSN. An explanation would be welcome. It seems to me this has been going on for a long time, and so far there's been nothing but an embarrassed silence.

[sem4u]

From looking at my stats it looks like it started happening on 16th/17th March.

[msndude]

Thank you for your patience. Our team is continuing the investigation. We have made some changes. Please let me know if you notice any changes (positive or negative) over the next few days.

Thanks,

Brett

[SEOPTI]

No changes at all, still heavy fake requests.

[sem4u]

It is still happening on my sites.

[Rosalind]

No change here, I'm still seeing it.

[blend27]

1 small site - less than 300 active pages.

6136 page downloads in the past 4 month by this thingy, last hit: 2009-04-26 18:04:23.680 EST

Average page weight 80KB

That is 490880 KB = HaFa GIG

I wonder how much time it will take an average "Pedro, The Realtor" to visit and read 6136 pages?

[SEOPTI]

Is there really a reason to hit a site with fake search requests for "phone" or "address" hundreds of times a day?

[edited by: SEOPTI at 10:24 pm (utc) on April 27, 2009]

[MS_Jason]

Hi All,
Thanks for updating us on what you're seeing. FWIW, we've got a post up on the webmaster blog on how to escalate these issues in the future: [blogs.msdn.com...]

Cheers,
Jason