homepage Welcome to WebmasterWorld Guest from 54.196.69.189
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe and Support WebmasterWorld
Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Microsoft launches Azure, an AWS competitor
Expect more unverifiable bots
incrediBILL




msg:3952684
 3:23 am on Jul 15, 2009 (gmt 0)


With the launch of Azure, Microsoft finds itself in a new type of business, where it competes with the likes of Amazon.com's Amazon Web Services

[news.cnet.com...]

The AWS [webmasterworld.com] service already plays host to a myriad of bad bots distributed randomly over AWS's IPs with no validation scheme [webmasterworld.com].

Many of us just block AWS completely to solve the problem.

A few more cloud computing situations like this and it will cause a serious rift between new technologies and sites blocking overwhelming bot abuse.

Wonder if Azure would be open to providing validation schemes for their clients?

IMO, depending on how it's implemented, Azure could cause some serious trust issues between webmasters and Microsoft.

 

wilderness




msg:3952701
 4:04 am on Jul 15, 2009 (gmt 0)

Wonder if Azure would be open to providing validation schemes for their clients?

You mean like MS has been doing at 131.107. since 2003 ;)

jdMorgan




msg:3952702
 4:05 am on Jul 15, 2009 (gmt 0)

Great. More 'bots with dynamic, un-ascribable IP addresses.

On the other hand, I already have 'serious trust issues' with Microsoft, ranging from the malformed MSN user-agent strings, to the "tide" hosts issuing their bogus search referrals, to the various past malfunctions in msnbot's handling of robots.txt and on-page robots metatags, to the lack of rDNS on many requests from their IP blocks... IMO, Bing works amazingly well considering the extreme 'sloppiness' of their infrastructure compared to other search providers.

If Azure's IP blocks are not well-partitioned away from other MS services, I will likely just pull the plug and block the lot of them; Google, Yahoo, and Ask traffic will just have to do -- as before.

Jim

GaryK




msg:3952765
 6:22 am on Jul 15, 2009 (gmt 0)

My trust issues with Microsoft have led me to block anything that looks like it's coming from them on all but one of my sites; I need to know what they're up to.

This will likely be one more thing to block. Just like I do with AWS. Unless they manage to implement some form of validation scheme--perhaps like the one Lord Majestic claims to be working on. But I think we all know how likely that is.

Bill, will we be permitted to post the Azure IP ranges as they become known to us so anyone who wants to can add them to their ban lists?

thetrasher




msg:3952878
 11:23 am on Jul 15, 2009 (gmt 0)

NetRange: 70.37.0.0 - 70.37.191.255
CIDR: 70.37.0.0/17, 70.37.128.0/18
NetName: MICROSOFT-DYNAMIC-HOSTING

dstiles




msg:3953191
 7:10 pm on Jul 15, 2009 (gmt 0)

Thanks for the IP range. Blocked. MS is bad enough at the best of times.

I see the /18 above that is unallocated: watch out for expansion?

incrediBILL




msg:3953280
 9:35 pm on Jul 15, 2009 (gmt 0)

Bill, will we be permitted to post the Azure IP ranges as they become known to us so anyone who wants to can add them to their ban lists?

I think thetrasher beat everyone to it! :)

Not sure that's the range they'll be using but it'll be easy to spot stuff bouncing off my firewall when it starts to happen.

GaryK




msg:3953424
 12:59 am on Jul 16, 2009 (gmt 0)

but it'll be easy to spot stuff bouncing off my firewall when it starts to happen.

I really should write a script to track stuff like that and incorporate it into my weekly report. I have no idea what UAs are getting blocked at the door based on IP Address bans.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved