This came in my mind when checking googlebot spider "signature". It is identified as googlebot and then we can see [googlebot.com...] in the same line. This is all in 1 line of logs. Similar is with fast and some other.
Now has anyone think of this: write a spider which will crawl other sites on the net by following the links. Simmilar like any normal spider. Then you could write in the code so your spider will identify itself with some kind of AD: yourcompanyspider - visit us at yourcompany.com (or some other words).
I know that very little people actually look on stats everyday but this spider could visit thousand sites daily. Some people will look -i know i check when i see newspider in my logs.
So is this considered spam? We didnt contact them by e-mail. Is sending "an unwanted" or better "not requested" spider a "criminal act"?
Are there any technical dificulties here. Did anyone think of this kind of promotion?:)
ps: if this is the wrong forum please move my post.thanks.
Been there - done that. Several years ago. 97 was my first running - 2000'ish my last. Currently, you'll find about 30-40 of the same thing running around on a regular basis. Most of them don't use agents, but just generate false referrers back to a site.
This falls under stealth marketing.
We have several members who have their browsers setup to generate false referrers on every page view. Some are quite creative and link to things like the click through counters at Yahoo or Direct Hit.
The problem with them, is that the results declined so far, that it got into 10k pages viewed to get one return user. When I did it the first time, it was like 500 to 1 (I thought I'd found the holy grail).
Well, there really is a variable embedded here that we have to take a look at.
If the bot advertisement rotates back to an actual search engine/directory service, most people won't be angry about it.. you're just another promotional listing for their web site.
If the spider leaves behind an advertisement which points back to your "web design" business, you'll probably enjoy some dDOS attacks for your efforts.
I mean, Google has spidered my sites and I've never submitted to it.. but I really don't take offense to this happening because Google is a real search engine. Now, if Google happened to be any other type of company.. I would be pretty annoyed.