WebSearchBench

no robots either

[bull]

129.217.16.140 - - [01/Aug/2003:16:17:24 +0200] "GET / HTTP/1.0" 200 2363 www.-.net "" "WebSearchBench WebCrawler v0.1(Experimental) ([e-mail])" "-"
129.217.16.141 - - [01/Aug/2003:16:19:34 +0200] "GET / HTTP/1.0" 200 2363 www.-.net "" "WebSearchBench WebCrawler v0.1(Experimental) ([e-mail])" "-"

from a german university. did not want robots.txt. Mailed them but go no response so far. Only / fethced.

jan

[bull]

fast response.

[websearchbench.cs.uni-dortmund.de...]

The person who responded says they obey the robots exclusion std, but I could not find any reference to it in my logs.