NutchCVS/0.03-dev_(Nutch;_http://www.nutch.org/docs/bot.html;_nutch-ag - Crawler, Spider, and User Agent ID forum at WebmasterWorld - WebmasterWorld

Forum Moderators: open

Message Too Old, No Replies

NutchCVS/0.03-dev_(Nutch;_http://www.nutch.org/docs/bot.html;_nutch-ag

To block or not to block.

requiem

5:19 pm on Sep 28, 2003 (gmt 0)

10+ Year Member

[edit]
Stupid question. I have now read the previous WW posts.
I am blocking this spider!
[/edit]

[edited by: requiem at 5:34 pm (utc) on Sep. 28, 2003]

pendanticist

5:24 pm on Sep 28, 2003 (gmt 0)

WebmasterWorld Senior Member

10+ Year Member

Yeah, it's [google.com] been mentioned before.

Pendanticist.