Forum Moderators: open
Does anyone have any information about this bot and who runs it? Is it malicious?
Convera.com is a leading provider of enterprise search and categorization solutions delivering categorization, dynamic classification, federated search, taxonomy development, taxonomy management and portal integrations for commercial and government customers around the world.
It appears to me as if they are spidering both for resale of their services and data to third parties?
What do I know!
From one of their partner sites:
"applications including enterprise portals, knowledge management, intelligence gathering, profiling, corporate policy compliance, regulatory compliance, customer service"
It appears to me as if they are spidering both for resale of their services and data to third parties?
Definately - they seem to be doing the same thing as Google, Yahoo, MSN and many others.
Definately - they seem to be doing the same thing as Google, Yahoo, MSN and many others.
<snip>
First, NONE of the three you mentioned above are spidering with the primary itenary of selling what they gather to a third party.
The aforementioned three offer public search engines of the data they gather. Covera offers no such free tool.
Second, Covera is not spidering my sites, which was effective on Monday the 18th.
[edited by: volatilegx at 3:11 pm (utc) on Oct. 25, 2004]
[edit reason] lets keep flames out of this [/edit]
Second, Covera is not spidering my sites, which was effective on Monday the 18th.
I think you should give them a call and point out that the value of publicly available data that they have spidered now dropped dramatically -- please feel free to do just that as otherwise they might not even notice that loss of value ;)
And please no need to use insults - I merely pointed out that businesses such as Google, MSN and Yahoo exist for the sole purpose of "spidering both for resale of their services and data to third parties" - these, and not the provision of free services such as public search, are the stated objectives and legal obligations to shareholders of respective organisations.
So, in other words, we don't have a consensus about whether or not these folks should be added to our "Bad Robots" .htaccess lists?
Any other opinions, folks?
we don't have a consensus
mattie,
your relatively new here.
As a rule this forum has not a whole either gathered or determined an consensus of good or bad.
Rather, it has been a tradition the each webmaster determines personally what bots are either beneficial or detrimental to their own sites (s).
The forum has in the past accepted some very off topic conversations in thread that were not part of the initial thread topic, rather solutions and/or preventions that culminated as a result of the same thread.
My own sites are quite unique. Many of the pages hold content which is not found any other place on the internet. As a result, the content of my pages are more crucal to a new or smaller bot than, that bot might provide benefits or visitors to my sites.
My htaccess denials (of APNIC, RIPE and LACNIC) are personal choices. Although I've had communication with other webmasters whom deny access to specific reigons.
Consensus? Hardly.
Today:
63.241.61.80
my bandwidth is going crazy
SetEnvIf User-Agent ^Convera keep_out
Each webmaster must decided what is beneficial or detrimental to their own website (s).
Don
Regards...jmcc
