Page is a not externally linkable
- Local
-- Foo
---- Attack of the Robots, Spiders, Crawlers.etc


AlexK - 8:13 pm on Nov 25, 2005 (gmt 0)


Brett_Tabke:
Seems like there is a great deal of interest in the topic
You bet - one of the most interesting I can remember (because it is live, and totally relevant to my site - that just has to also echo out to others).

It was a, throw hands in the air I've had it moment
That is the human element that gets us all coming back here, Brett. Give me human over spiders every time.

rogue bots are the #1 issue we face
Another echo of a problem we all face [webmasterworld.com] - the link is the way I've managed to fix it on my site (and how I found WebmasterWorld in the first place), and it is so useful to know the problems faced here, and how you are managing it.

On the issues of Search - since G is so good at this, do they have a solution that could be installed on your server? Just a thought.

Added:
waynet:
I was also thinking one of those Google search appliances...
Great minds, huh?

[edited by: AlexK at 8:16 pm (utc) on Nov. 25, 2005]


Thread source:: http://www.webmasterworld.com/foo/9618.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com