I've been doing my best to kick to the curb as many bots that "don't make me money" as possible. You know, all the bots that are about collecting data for back-link analyzers / SEO tools, competitive analysis bots, etc.
My question is, beyond Googlebot, Bingbot or Slurp which bots actually do publishers any good? Some of the bots in the "ad placement analytics" space are relentless, hitting page after page, again and again. This can slow a website or server AND, as we have been told, speed is a factor in ranking and ranking affects traffic which affects income . . . soooooooo . . bot . . show me the money or I'll show you the door.
Which bots do you "let in" and which do you kick to the curb?
Has anyone "played around with this" and have any insight to share? Did you ban bots x,y & z (remember, I'm NOT talking about G/B/Y bots) and did that affect your income?
Wouldn't it be nice if the geniuses running these bots actually shared some of their data with publishers to demonstrate how they are actually doing "us publishers" some good. "Share data"? Yeah, as in "We identified Site X. Based upon our data we recommended to our client C that they target Site X for ad placements / targeting. Site X reaped the following benefit." Yeah, like <that.