homepage Welcome to WebmasterWorld Guest from 54.243.12.156
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Potbot not respecting robots.txt
Scooter24




msg:1527005
 12:32 pm on Jan 24, 2003 (gmt 0)

Maybe somebody is interested in this:

I had a visit by Potbot 1.0 (IP: 216.40.238.201). It read the robots.txt file and then proceeded to browse through directories which were disallowed (automatically getting banned in the process).

 

korkus2000




msg:1527006
 1:09 pm on Jan 24, 2003 (gmt 0)

We had a discussion on potbot earlier.
[webmasterworld.com...]

little guy must be in full server abuse mode.

Scooter24




msg:1527007
 1:59 pm on Jan 24, 2003 (gmt 0)

I know, but so far nobody has reported abuses by potbot (which is why I made this post)

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved