Forum Moderators: open

Message Too Old, No Replies

AntBot

         

lucy24

6:11 pm on Oct 22, 2023 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



UA: AntBot (and-that’s-all)
IP: The Usual Suspects
Started showing up earlier this week. To date have made well over 200 requests for robots.txt, in each case receiving the minimalist
User-Agent: *
Disallow: /
because their name fits a bad_agent criterion.

On their first visit, they requested a few pages--blocked, of course--but other than that it's been nothing but the same robots.txt over and over.

I feel no urge to inquire more deeply.