Bot coming from Microsoft IPs

[Mokita]

It doesn't ask for robots.txt and has blank UA and blank referrer.

Asked for all the html pages in one of our sites but got 403s as I don't allow blank UAs.

It started out like this for five pages:

131.107.0.86 - - [22/Jan/2007:08:17:44 -0500] "GET / HTTP/1.1" 403 - "-" "-"

... then changed to this IP a few hours later:

207.46.89.16 - - [22/Jan/2007:11:30:40 -0500] "GET /page.html HTTP/1.1" 403 - "-" "-"

I've seen mention in this forum of Microsoft doing this some years ago. Does anyone know what they are looking for?

[webmasterworld.com...]

[wilderness]

Mokita,
MS has so many coals in the fire, who knows what that are up to.

This IP range has long been a host for testing new schemes.

Don

[Mokita]

Thanks for the reply Don.

It seems a bit self-defeating using a blank UA for whatever they are testing. There must be quite a few sites they'd be banned from.

I didn't really expect that anyone would know what they are looking for - but something might have been uncovered/discovered since that old thread I posted.

I guess I also wondered if they are checking for cloaking and that giving them a 403 might result in penalties by MSN or Live Search.

[wilderness]

Mokita,
There were two other simultaenous threads around the same time as the previously mentioned..

This one quite quite long
[webmasterworld.com...]

The other thread, I'm recalling was around four pages however, I've never had any luck locating that thread in the archives.

[incrediBILL]

Some IPs in 207.46.89.* appear to be a proxy "1.1 JPN-PRXY-01" and appears to possibly be related to the WPFE (Windows Presentation Foundation “Everywhere”)

Some IPs in 131.107.0.* also appear to be a proxy with "1.0 RED-PRXY-30", "1.1 NET-PRXY-03" and "1.1 SEA-PRXY-01" but I don't know what it's all for. The blank user agent was sequentially crawling my site. I've also seen "Java/1.5.0_06", "msnbot/1.0 (+http://search.msn.com/msnbot.htm)", "SandCrawler - Compatibility Testing", coming from this range.