MSIndianWebcrawl

Forum Moderators: open

Message Too Old, No Replies

MSIndianWebcrawl

Pfui

9:54 pm on Jun 12, 2009 (gmt 0)

209.133.64.nnn
MSIndianWebcrawl

robots.txt? YES

IP resolves to (CUE KNOWING GROANS) --
Abovenet San Francisco

wilderness

11:14 pm on Jun 12, 2009 (gmt 0)

Unfortunately they've chosen one of those taboo names in the UA.

209.249.11.4 - - [12/Jun/2009:23:22:59 +0100] "GET /robots.txt HTTP/1.0" 200 4777 "-" "MSIndianWebcrawl"

Pfui

5:56 am on Jun 13, 2009 (gmt 0)

209.249.11.3
MSIndianWebcrawl

robots.txt? YES

209.249.0.0 - 209.249.255.255 (209.249.0.0/16) is Microsoft Research.

MS in the UA notwithstanding, the IP in the OP cloaked their ID. Net result? NO.

GaryK

5:16 am on Jun 14, 2009 (gmt 0)

It hit my browser project site from Abovenet last week too. Took robots.txt and left.

I wonder, how does one crawl like an Indian?

incrediBILL

6:38 am on Jun 14, 2009 (gmt 0)

Perhaps it's a project from MS in India.

thetrasher

11:04 am on Jun 14, 2009 (gmt 0)

FYI: Former discussion about 209.249.11.0/28:
[webmasterworld.com...]

"hole-dweller"

209.249.0.0 - 209.249.255.255 (209.249.0.0/16) is Microsoft Research.

No:
[ws.arin.net...]

Pfui

4:32 am on Jun 16, 2009 (gmt 0)

My CIDR info was too broad; I didn't read all the way down the WHOIS page. (slaps head) Microsoft Research uses Abovenet but not all of 209.249. As of this moment at least, in addition to other hosts/IPs, it's:

209.249.11.0 - 209.249.11.15 (209.249.11.0/28)

IP Address: 209.249.11.3
NetRange: 209.249.0.0 - 209.249.255.255
CIDR: 209.249.0.0/16
OrgName: Abovenet Communications, Inc
---
Microsoft Research ABOV-T406-209-249-11-0-28 (NET-209-249-11-0-1)

Again, regardless, IPs cloaked their IDs x2. Not OK, imho.

keyplyr

7:11 am on Jun 16, 2009 (gmt 0)

My log history shows spoof of MSR-ISRCCrawler coming from Abovenet: 209.133.64.214