Errors - What actually happens?

[keyplyr]

Most of what shows up in my error_logs aren't really errors at all, but requests and/or disallowed methods from IPs or UAs that I have blocked for one reason or another.

This leaves what I consider to be actual errors. Almost always, these are either failed requests for files resulting from omitting the sub-directory, or failed requests for files resulting from adding an incorrect sub-directory. This has always happened, for years now with different sites, different servers.

How does this happen? Does a browser become confused? Do data packets get mixed-up with each other, maybe by fragmenting? Is it the server?

[paybacksa]

You actually look at your error logs?

Just kidding! :-)

Many hits for improper subdirs (variants or just abctype stuff) are generated by bots scanning for stuff you may not have published. It's SOP when scanning to try for all the possibles, not just what is linked.

If I were the type who ran harvesters against peoples web servers, I would probably program it to try several variations of every directory name you link through to, just to be sure I got it all. Of course, I don't do that, but some people do.

[jdMorgan]

In no particular order, some of the causes could be:

Broken robots

Malicious robots

Bad links posted on other sites

Bad links posted on your own sites

Incorrectly typed-in URLs

Curious people "trying" different URLs to see what happens

Ne'r-do-wells "trying" different URLs to see what happens

You only have to worry about your server if you see errors on *good, valid* URLs requested by user-agents and IP addresses that you *have not* blocked. In that case, you'd have to question why the server reported an error. TCP/IP has several built-in "checks" to make sure that incomplete and corrupted packets are retransmitted until received correctly.

Jim