| Potbot not respecting robots.txt
|
Scooter24
msg:1527005 | 12:32 pm on Jan 24, 2003 (gmt 0) |
Maybe somebody is interested in this: I had a visit by Potbot 1.0 (IP: 216.40.238.201). It read the robots.txt file and then proceeded to browse through directories which were disallowed (automatically getting banned in the process).
|
korkus2000
msg:1527006 | 1:09 pm on Jan 24, 2003 (gmt 0) |
We had a discussion on potbot earlier.
[webmasterworld.com...] little guy must be in full server abuse mode.
|
Scooter24
msg:1527007 | 1:59 pm on Jan 24, 2003 (gmt 0) |
I know, but so far nobody has reported abuses by potbot (which is why I made this post)
|
|
|
