findlinks bot

130.83.167.153

[Bewenched]

findlinks/2.6+(+http://wortschatz.uni-leipzig.de/findlinks/)

[keyplyr]

what about it?

[lucy24]

It visits me every few days. Not sure what it wants, but I hate to think it could be actively malign. I mean, Leipzig, they've been around forever ...

Ziel von FindLinks ist die Beschaffung der Datengrundlage für NextLinks. Dazu werden aus möglichst vielen HTML-Seiten (zunächst aus den Domänen .de, .at und .ch) die darin auftretenden Links analysiert.

Uh... they've used up all of Germany, Austria and Switzerland and are now scraping the barrel of dot com?

Die Datei robots.txt wird vom FindLinks-Server berücksichtigt. Änderungen in einer solchen Datei wirken sich nach spätestens ca. 30 Tagen aus.

Oh, wait, I think I've read that before. 30 Tagen?! Are they kidding? Even the googlebot doesn't go much past 24 hours. They're exaggerating anyway. Quick detour to logs suggests that they alternate between robots.txt and some other request, so it can't be more than a few days. Why they even bother with the separate crawls is anyone's guess.

[keyplyr]

I've denied them via robots.txt for years. So far they've always obeyed and stayed away from my files.