Why do many rougue bots NOT fake their UA?

This makes no sense.

[mikomido]

Daily I find 2-3 bots that I add to my blocklist after seeing them looking around and then looking them up on Google and seeing references to spam/harvesting.

WHY do these bots not fake MSIE's User-agent? That would make me unable to block them.

[vincevincevince]

By the time you found them, they'd got what they wanted

[mikomido]

Well, they can't ever come back (and expect me to return content). So why not fake MSIE's UA? It would fool me for sure.

[vincevincevince]

How big is the web? Even if a few thousand sites block them, they still have plenty more content to scrape.

[mikomido]

Again... why expose themselves at all? It just makes no sense.

[vincevincevince]

My thoughts:
1) They are too lazy / couldn't be bothered to read the manual
2) By allowing you to stop them easily it makes you less likely to cause a fuss about them to their ISP, etc.

[jatar_k]

it makes it easy for them to see who actively blocks too. They can come back later with an MSIE agent if needed.

plus, 'rogue' is not the same thing as 'bad'.

A rogue bot could be a bot from a good source that is having a problem, I think you are talking mainly about 'bad' bots

[mikomido]

Based on the User-agent I can see if it's a known bot with "problems". I don't want any bots other than large search engines.