Forum Moderators: open
We do get crawled regularly from something like crw54.vc.search.re4.yahoo.com, but the ext9.vc.search.re is a mystery. There was one log entry today from ext9.vc.search.re2.yahoo.com which looked at robots.txt (no referrer info: GET /robots.txt HTTP/1.1" 200 271 "-" "-")
NO log entries showed up under this name (e.g. for GET / POST, etc), but our web site came to a screeching halt because of all the connections.
If the offending IP is re-enabled, the same things happens: we get buried under https connections.
Anybody have suggestions for next-steps?