Forum Moderators: open
Took main page and graphics.
No robots.
The link in the UA does not work, however if you back up to the subfolder, there's plenty to read (except exclusion)
67.195.50.98 - - [15/Sep/2009:01:40:49 +0100] "GET / HTTP/1.1" 200 5904 "-" "Mozilla/5.0 (compatible; Yahoo! SearchMonkey 1.0; [developer.yahoo.com...]
Mozilla/5.0 (compatible; Yahoo! SearchMonkey 1.0; [developer.yahoo.com...]
67.195.50.120
hlf330028.crawl.yahoo.net
-----
OrgName: Yahoo! Inc.
OrgID: YHOO
NetRange: 67.195.0.0 - 67.195.255.255
-----
ROBOTS.TXT? No
When it came a' crawling a few weeks ago, it only asked for/retrieved a single 1x1 Project Honey Pot bot trap near the end of a home page. Odd how that's the only thing it went for, although I'm glad it took the bait.
hlf330027.crawl.yahoo.net
Mozilla/5.0 (compatible; Yahoo! SearchMonkey 1.0; [developer.yahoo.com...]
robots.txt? NO