log headers from robots.txt

[lucy24]

Keeping in mind that I only speak about three words of php...

What's the most painless way to log headers on a robots.txt request? Currently I've got a logheaders.php that's part of the general footer, so it's invoked whenever the server builds a page (including, by design, the 403 page, so headers will be logged on all blocked requests, even non-pages that I normally can't be bothered with). But obviously you can't Include anything in a plain-text file.

I thought of rewriting to a "robots.php" which would consist of two things: logheaders.php and the requested robots.txt. But then it all turns into run-in html onscreen and no longer looks like a plain-text file to the naked eye. It does still look like plain text in the html source, so is this actually a non-problem as far as robots are concerned?

Or am I on the wrong track entirely? (Probably...)

[robzilla]

The difference lies in the Content-Type header. Your web server will push "robots.php" with text/html as the content type, which is different from the text/plain content type of a simple text file like robots.txt (even though, with the rewrite, the URI hasn't changed).

Thankfully, you can override any header using the header() function. Note that it has to be called before your script sends any output:

header('Content-Type: text/plain');

Reload /robots.txt and its contents should appear in the familiar monospaced font.

Nothing wrong with the method, in my opinion. I would probably do the same.

[lucy24]

Tralala, that worked beautifully!

Thanks :)