Raw logs - handling special characters

[Umbra]

Our unix server writes access logs in this format:

domain¦file path¦log entry

...so that there are 3 columns delimited by the pipe symbol

Now we've found that certain hack simulations can somehow insert pipe symbols into the file path, and of course this causes problems for log software which tries to split each line into 3 columns using pipe as a delimiter. Even newline characters can be inserted, resulting in an empty blank line in the logs.

Shouldn't the server be encrypting special characters before writing them into the log? Or is it industry standard for Linux servers to just the log data exactly as is?

[gregbo]

I checked RFC 3986, and the vertical bar is not a reserved character. In general, I think you should use horizontal tabs as field delimiters in the logs. If a client sends a horizontal tab as part of a URI, it should be pct-encoded (e.g. %09, with 09 the hex value for a horizontal tab), and should be logged as such by the server.

[Umbra]

Thanks Gregbo...

Is there anyone else whose server writes log data like that (3 columns delimited by a pipe symbol) or is our server just using a totally non-standard system?

[gregbo]

Offhand, I don't know of any that use vertical bars. I'd think that this feature could be changed via software if not by configuration. (Especially if it's open source.)