Forum Moderators: open
209.131.41.19 - - [10/Nov/2004:17:48:27 -0800] "GET /robots.txt HTTP/1.0" 200 1705 "-" "Mozilla/4.0 (compatible; MSIE 5.0; Windows NT)"
209.131.41.19 - - [10/Nov/2004:17:48:27 -0800] "GET /Blahblah.html HTTP/1.0" 200 12310 "-" "Mozilla/4.0 (compatible; MSIE 5.0; Windows NT)"
Would that be the case here, too? As you can see, it requested robots.txt and a single file.
[11/10/04 19:19:59 IP block 209.131.41.19
Trying 209.131.41.19 at ARIN
Trying 209.131.41 at ARINOrgName: Yahoo
OrgID: YHOO
Address: 701 First Avenue
City: Sunnyvale
StateProv: CA
PostalCode: 94089
Country: USNetRange: 209.131.32.0 - 209.131.63.255
CIDR: 209.131.32.0/19
NetName: FC-YAHOO1
NetHandle: NET-209-131-32-0-1
Parent: NET-209-0-0-0-0
NetType: Direct Allocation
NameServer: NS1.YAHOO.COM
NameServer: NS2.YAHOO.COM
NameServer: NS3.YAHOO.COM
NameServer: NS4.YAHOO.COM
NameServer: NS5.YAHOO.COM
Comment:
RegDate: 1999-07-09
Updated: 2004-02-23OrgTechHandle: NA258-ARIN
OrgTechName: Netblock Admin
OrgTechPhone: +1-408-349-3300
OrgTechEmail: netblockadmin'at'yahoo-inc.com# ARIN WHOIS database, last updated 2004-11-10 19:10
# Enter? for additional hints on searching ARIN's WHOIS database.
Is the act of requesting robots.txt significant in some way?
<Good thing I checked this IP Number as those that request robots.txt (without a declarative UA) are usually looked upon as potentially dangerous and summarily banned.> :)