Welcome to WebmasterWorld Guest from 54.144.126.195

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

findlinks bot

130.83.167.153

     

Bewenched

2:03 am on Dec 4, 2012 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



findlinks/2.6+(+http://wortschatz.uni-leipzig.de/findlinks/)

keyplyr

5:47 am on Dec 4, 2012 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



what about it?

lucy24

6:46 am on Dec 4, 2012 (gmt 0)

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



It visits me every few days. Not sure what it wants, but I hate to think it could be actively malign. I mean, Leipzig, they've been around forever ...

Ziel von FindLinks ist die Beschaffung der Datengrundlage für NextLinks. Dazu werden aus möglichst vielen HTML-Seiten (zunächst aus den Domänen .de, .at und .ch) die darin auftretenden Links analysiert.

Uh... they've used up all of Germany, Austria and Switzerland and are now scraping the barrel of dot com?

Die Datei robots.txt wird vom FindLinks-Server berücksichtigt. Änderungen in einer solchen Datei wirken sich nach spätestens ca. 30 Tagen aus.

Oh, wait, I think I've read that before. 30 Tagen?! Are they kidding? Even the googlebot doesn't go much past 24 hours. They're exaggerating anyway. Quick detour to logs suggests that they alternate between robots.txt and some other request, so it can't be more than a few days. Why they even bother with the separate crawls is anyone's guess.

keyplyr

8:00 am on Dec 4, 2012 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



I've denied them via robots.txt for years. So far they've always obeyed and stayed away from my files.
 

Featured Threads

Hot Threads This Week

Hot Threads This Month