Forum Moderators: open

Message Too Old, No Replies

UnChaos

does not follow robots.txt

         

notriddle

6:06 pm on Jan 9, 2020 (gmt 0)

5+ Year Member



IP: 141.226.169.nn
UA: <a href='http://www.unchaos.com/'> UnChaos </a> From Chaos To Order Hybrid Web Search Engine.(vadim_gonchar@unchaos.com)
robots.txt: NO


Unlike the report back [webmasterworld.com...] I haven't observed them ever requesting robots.txt. And, in fact, they've requested pages that I have roboted-out.

lucy24

6:14 pm on Jan 9, 2020 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Is that <a href> business a part of the literal UA string?

Hahahahaha.