Forum Moderators: open

Message Too Old, No Replies

OmniExplorer Crawler

Should I continue to let this spider hammer my sites?

         

FlipFlops

3:21 pm on Aug 16, 2005 (gmt 0)



OmniExplorer Crawler has been pulling pages off of my web site to rats as high as 22,000 requests per day. I've checked out the references to this crawler on _www.omni-explorer.com_ but I have a hard time letting spiders in who promise to refer users when there's no particular association of the spider to an engine.

Does anyone else have experience with OmniExplorer Crawler? Any value to it thus far? Should I disallow it?

* Apologies if this has been recently posted. I tried the site search, but the search system was down at time of posting. *

wilderness

9:36 pm on Aug 16, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



There's old threads on this.
One of which is fairly recent:

[google.com...]

site search

[webmasterworld.com...]

add forum11 to your search box and the +name

mcneely

6:10 am on Aug 20, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



OmniExplorer Crawler

Just another handy dandy spam bot from HE as far as I'm concerned.

IxNayed it eons ago