IP: various Usual Suspects, especially in 34 and 136
UA: Mozilla/5.0 (compatible; Reflectionbot/1.0; +https://reflection.ai/bot)
robots.txt: yes but...
Started showing up just a few weeks ago, in mid-August 2026.
Asks for robots.txt frequently. After I Disallowed it, an interesting behavior became visible. When the robots.txt request receives a 200, finding itself Disallowed by name it goes away for a while. When instead it receives a 429 (from the server, “too many requests”, I think) it immediately requests a page--which will also get a 429. I believe the reasoning is “Well, I
tried to get robots.txt ::whine:: but all I got was an error, so I guess it’s OK to request a page.” (Any random interior page, incidentally, so it’s getting a shopping list from someone else.)
Oh, and the URL in the UA leads directly to a 404. Figures.
