Weberknecht

[lucy24]

I initially thought this was just another ebook scraper (these are real directories):

91.64.54.*** - - [04/Feb/2016:07:46:58 -0800] "HEAD /ebooks/ramona/ HTTP/1.0" 200 268 "-" "Weberknecht" 
91.64.54.*** - - [04/Feb/2016:07:48:36 -0800] "HEAD /ebooks/egede/ HTTP/1.0" 200 245 "-" "Weberknecht" 
91.64.54.*** - - [04/Feb/2016:08:11:36 -0800] "HEAD /ebooks/alcott/ HTTP/1.0" 200 268 "-" "Weberknecht" 
91.64.54.*** - - [04/Feb/2016:08:12:04 -0800] "HEAD /ebooks/bc/ HTTP/1.0" 200 269 "-" "Weberknecht" 

but fortunately it offended me just enough that I blocked the UA. I say "fortunately" because it has since returned, making a total of over 1000 requests to date. (This may not sound like a lot, but it's far more than the number of pages on the site.) Based on timestamps, they're still hammering away. I blocked them by UA (German for, I guess, "apprentice spider") because I think the IP is human broadband.

91.64.54.*** - - [04/Feb/2016:15:54:48 -0800] "GET /ebooks/alcott/ \"/boilerplate/legal.html\" HTTP/1.0" 403 3397 "-" "Weberknecht" 
91.64.54.*** - - [04/Feb/2016:15:54:49 -0800] "HEAD /ebooks/alcott/ \"/boilerplate/./deed.fr HTTP/1.0" 403 212 "-" "Weberknecht" 
91.64.54.*** - - [04/Feb/2016:15:54:50 -0800] "HEAD /ebooks/alcott/ \"/boilerplate/./deed.id HTTP/1.0" 403 212 "-" "Weberknecht" 
...
91.64.54.*** - - [05/Feb/2016:03:57:34 -0800] "HEAD /wiki/Wikipedia:Lokal_K HTTP/1.0" 403 212 "-" "Weberknecht" 
91.64.54.*** - - [05/Feb/2016:03:57:34 -0800] "HEAD /wiki/Spezial:%C3%84nderungen_an_verlinkten_Seiten/Vorlage:Navigationsleiste_Regionale_Treffpunkte_der_Wikipedianer HTTP/1.0" 403 212 "-" "Weberknecht" 
91.64.54.*** - - [05/Feb/2016:03:57:34 -0800] "HEAD /wiki/Hilfe:Neuen_Artikel_anlegen HTTP/1.0" 403 212 "-" "Weberknecht" 
91.64.54.*** - - [05/Feb/2016:03:57:35 -0800] "HEAD /w/index.php?title=Vorlage:Navigationsleiste_Regionale_Treffpunkte_der_Wikipedianer&action=info HTTP/1.0" 403 212 "-" "Weberknecht" 
...
91.64.54.*** - - [06/Feb/2016:02:59:20 -0800] "GET /ebooks/alcott/ \"/boilerplate/ \"/ebooks/wedding/\" HTTP/1.0" 403 3397 "-" "Weberknecht" 
91.64.54.*** - - [06/Feb/2016:02:59:24 -0800] "HEAD /ebooks/alcott/ \"/boilerplate/ \"/ebooks/wedding/javascript:BrowseByMenu('myTopic','/menu/topic/L_2010.htm','topic_l'); HTTP/1.0" 403 212 "-" "Weberknecht" 
91.64.54.*** - - [06/Feb/2016:02:59:24 -0800] "HEAD /ebooks/alcott/ \"/boilerplate/ \"/ebooks/wedding/javascript:BrowseByMenu('mySpeaker','/menu/speaker/KL_2010.htm','speaker_kl'); HTTP/1.0" 403 212 "-" "Weberknecht" 
91.64.54.*** - - [06/Feb/2016:02:59:25 -0800] "HEAD /ebooks/alcott/ \"/boilerplate/ \"/ebooks/wedding/javascript:BrowseByMenu('myTopic','/menu/topic/S_2010.htm','topic_s'); HTTP/1.0" 403 212 "-" "Weberknecht" 
91.64.54.*** - - [06/Feb/2016:02:59:28 -0800] "HEAD /ebooks/alcott/ \"/boilerplate/ \"/ebooks/wedding/javascript:BrowseByMenu('myLocation','/menu/country/CE_2010.htm','country_ce'); HTTP/1.0" *** 212 "-" "Weberknecht" 
91.64.54.*** - - [06/Feb/2016:02:59:29 -0800] "HEAD /ebooks/alcott/ \"/boilerplate/ \"/ebooks/wedding/javascript:BrowseByMenu('myTopic','/menu/topic/G_2010.htm','topic_g'); HTTP/1.0" 403 212 "-" "Weberknecht" 

Et cetera, et cetera. I've just pasted in some random chunks to give the flavor of the thing.

Ah, malice and stupidity, always a winning combination.

[edited by: keyplyr at 10:38 am (utc) on May 26, 2016]
[edit reason] depersonalized IP address [/edit]

[keyplyr]

Running from a home cable account. Usually these don't live too long.

[lucy24]

I should also mention that you can spot this agent by a consistently malformed header:

Content-Type: text/plain; charset=utf-8Cache-Control: max-age=0

[sic, over hundreds of visits spanning many days, so not a logging hiccup]

[keyplyr]

malformed header

Likely due to that Zika virus.

[keyplyr]

Still alive from same IP doing HEAD requests on HTML pages.