| Edacious & Intelligent Web Robot, Part 2
|
GaryK
msg:3154393 | 5:30 pm on Nov 12, 2006 (gmt 0) |
The previous thead [webmasterworld.com] on this user agent is locked to new replies. It's returned with a wacky new user agent: Mozilla/4.0 (compatible; MSIE is not me; EDI/1.6.6; Edacious & Intelligent Web Robot; Daum Communications Corp., Korea)
211.115.109.nn
No PTR
|
keyplyr
msg:3154854 | 6:50 am on Nov 13, 2006 (gmt 0) |
Yes, took 2 dozen pages - same new UA.
|
abates
msg:3155833 | 3:17 am on Nov 14, 2006 (gmt 0) |
Atleast they've fixed the robots.txt bug I noticed last time, though they're still requesting robots.txt for every page they're downloading (they could at least implement If-Modified-Since for it...)
|
|
|
