Requests threw a 406 error

User agent: "Mozilla/5.0"

[Mokita]

This is very odd - never seen a 406 error in the site previously. It only asked for 2 files:

209.8.20.nnn - - [17/Oct/2006:08:32:41 +1000] "GET /file1.htm HTTP/1.1" 406 281 "http://google.com" "Mozilla/5.0"
209.8.20.nnn - - [17/Oct/2006:08:32:45 +1000] "GET /file2.htm HTTP/1.1" 406 276 "http://google.com" "Mozilla/5.0"

IP belongs to "Beyond The Network America, Inc."

[Ocean10000]

406 Not Acceptable [w3.org]

The resource identified by the request is only capable of generating response entities which have content characteristics not acceptable according to the accept headers sent in the request.

Unless it was a HEAD request, the response SHOULD include an entity containing a list of available entity characteristics and location(s) from which the user or user agent can choose the one most appropriate. The entity format is specified by the media type given in the Content-Type header field. Depending upon the format and the capabilities of the user agent, selection of the most appropriate choice MAY be performed automatically. However, this specification does not define any standard for such automatic selection.

Note: HTTP/1.1 servers are allowed to return responses which are
not acceptable according to the accept headers sent in the
request. In some cases, this may even be preferable to sending a
406 response. User agents are encouraged to inspect the headers of
an incoming response to determine if it is acceptable.

If the response could be unacceptable, a user agent SHOULD temporarily stop receipt of more data and query the user for a decision on further actions.

[Mokita]

Thanks for the info Ocean :)

Note that the two html files requested are viewed by normal browsers and legitimate bots every day, no errors evident.

I think this is a malformed bot of some kind.

[Ocean10000]

My guess is that this Bot (I assume it was a bot). Didn't send a well formed Accept header.

The following is an example Accept header, with a wild card for the last item, in short saying it accepts everything.

Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-powerpoint, application/vnd.ms-excel, application/msword,*/*

My guess is that the bot didn't send a Accept header that would allow the server to send text/html as a response mime type so your server answered with a 406 error, saying it could not respond with something it could understand.