Tracking the 404 pages

how to track which page requests generated 404 error

[vipink]

I recently renovated my website with a lot of new content and old pages were renamed to much extent (though old pages are still in search engine rankings). When I see my analytics report, I found that my 404 page has been among the top 5 visited pages.

I have set 404 error document to a certain "page_not_found.php" page. Now I want to know what are those pages that generated 404 error (ie those pages that added up to the visits to page_not_found.php). Though I have already done .htaccess redirects to some of the old pages but i don't have list of ALL the old page so I cant redirect then accordingly.

Kindly suggest some solution so that I may identify where my visitors stucked and direct then to appropriate content.

[phranque]

you should be able to see which pages referred you to the custom 404 page by checking the server access log.

by the way, make sure you are actually serving a 404 code with your custom page!

[vipink]

by the way, make sure you are actually serving a 404 code with your custom page!

Can you please tell me how to do that?

[Receptional]

In answer to the original question,

Put a tracking tag on the php page which records the referrer in a text file somewhere every time it loads. That will save you ploughing throgh log files.

If that's complicated, bung a freebie tracking counter on that page all on its own - one that tracks referring URLs. See the list of choice in this forum's library.

[phranque]

answering vipinks'?:

Can you please tell me how to do that?

according to the HTTP/1.1 specification (http://www.w3.org/Protocols/rfc2616/rfc2616-sec6.html),
the very first line of an http response message is the status line.
in this case you would use

HTTP/1.1 404 Not Found

the trick is finding out where this line is actually being generated for your server.

[LunaC]

The easiest way (for me at least) to check that the page is serving a propper 404 is with the Firefox extension Live HTTP Headers. Just remember to clear the cache before testing to get a clear result.

As for finding which pages are getting 404's, the referrers etc., I usually go with the easiest and add a free tracker to the custom 404 as well, just seems a lot simpler that way.