Welcome to WebmasterWorld Guest from 126.96.36.199
I've got a few collected from the forum here, but I just thought maybe someone had a full list of these rogue unwanted bots.
At a minimum, do you know these 4:
Aside from blocking them, what kind of "fun" can be had via cloaking with these bots?
Website monitoring services take users. Why would a user visit the site if they can "monitor" it from elsewhere. That defeats mission critical branding, it defeats promotion efforts, it defeats advertising, and that defeats your sites goals.
When a user visits your site and does not find updated content, they may find content or advertising they have not been exposed too. It's like channel surfing - it is how visitors are exposed to content they may not have seen before.
On the technical side, if people don't visit a site, it also means you are not counted by page counters such as the search engine toolbars like Google, Yahoo, and Alexa. That in turn may hurt your search engine rankings.
By not actively blocking these monitors, you are allowing and endorsing the poaching of your visitors. Website monitors are worse than Gator too me. Atleast with Gator, they have to visit your site.
Aside from the approved bots and partnerships with search engines, we do not allow unauthorized programmed querying of the site.
You may be right that some people get tricked into using some of these services (like some of the Gator examples) but with KaZaA, Lycos personalization etc, no, thats services and software people choose because they want to and because it helps them to make the web more usefull to them.
I don't think we can stop this "move" - not even if we all agree to do so here at WebmasterWorld - just like the music business can not stop P2P or MP3 files all together.
I truely understand the concerns that some might have with overuse of bandwidth, not being able to show ads etc. but let's deal with it instead of doing what the music business have been doing so far... I don't think they have handled it so well ;)
If users really feel a strong need for "off-browser" monitoring of the web and if "personalized agents" is here to stay then why not try and get the best out of it. I am pretty sure we can find a way to adopt to this and make money ... be creative. I thought we where the ones to be so :)
I don't believe in "the free web". It takes money to make good websites - even community sites like WebMasterWorld. I am not saying I want everything to be free - not at all. Also, I do undertsand it will take a lot of changes to profit from the clipping services, but I am very sure it will come. There will be be people that will understand how to develop websites that "fit" this format and make money from it.
Wasen't that we all did when we started SEO'ing back in the mid 90's? :)
Most people don't even know it happened. Imagine if the gas pump shorted you three spoon fulls on each tank for your life time. You don't "experience" the loss but it does add up over time. It reminds me of the cashier who took 1 penny out of every transaction for 40 years. No one could understand how she did it when she retired as a millionaire.
Web clippings and monitor services take a little bit of your site with them every time they connect. This is especially true if your site already has a update newsletter.
I think it was somewhere here I read that some infringement services gather data first then try to sell it. I believe this to be true due to the fact that my site mentions no less than 10 of the Fortune 25 companies. Most in not such a flattering manner, and although I have banned several infringement services, I have yet to hear from one of the Fortune 25 companies. Of course Iím a really small fish. My site has been up for 3 years.
You can add
deny from 12.40.85.