404 Errors with old images

missing images generate large amounts of 404 errors

[jarvis]

My site has recently been redesigned and i'm now receiving large amounts of 404 errors. The old site was predominantly image based so the 404's that are being returned are due to images that no longer exist.

I've set up custom error pages and receive email notification when an error is generated which is working like a treat. The error page emails me the requested page and the referring url.

My problem is that the majority of error emails i receive are for images that have been removed and there is no referring url.

I've set up several 301 redirects for the pages that were being requested but would prefer not to have to do the same for all of the old images as there would be hundreds.

Thanks in advance

[Dijkgraaf]

Is it users requesting it or is it bots/spiders?
I suspect it is probably the bots/spiders, or people visiting cached copies of the pages that had those images.

So what do you want to achieve?

To receive less e-mails?
You could change your custom error page so that if the request is for an image, and there isn't a referrer, don't send an e-mail.

To have less 404 errors in your log?
You could try adding disallows for the old images into your robots.txt, of course this is only practical you don't have to specify each one individually.

To get the people to come to your new page?
For the most commonly requested images that no longer exist, create another image with the same name, that has some text in it of the form "sorry that image no longer exists, please visit http://www.example.com/page.htm to see the new images"

[Edit to fix some spelling mistakes]

[edited by: Dijkgraaf at 6:22 am (utc) on Sep. 7, 2005]

[larryhatch]

Jarvis: What URLs exactly are calling for this obsoleted images?

Are they onsite? If so, fix them.
Are they offsite? Maybe your images are being hot-linked.
If the latter, dig thru your access logs and find out who exactly is doing all this
and take it from there. Best - Larry

[jarvis]

The problem is that I don't know what the requesting urls are?

I'm using a php script to report the errors via email which defines the two variables as below.

$referring_url = $_SERVER['HTTP_REFERER']; 
$requested_url = $_SERVER["REQUEST_URI"];

I assumed that the 'referrer' would be recorded in all cases except for spiders? which would sort themselves out after a little while as the spiders crawl the site and re-index it correctly.

The images in question are incidental... background images, spacers, etc... not product images / logos / avatars, etc.

I was originally wanting to stop the email if the error was generated for an image however that would also pose problems if legitimate links to images are broken.

Have to think about what solution will best suit?
Post back tomorrow.

[mattglet]

I don't know the PHP equivalent, but you should track the User Agent of the request. If it is in fact search engines, you can make it so you don't get error messages if they are caused by a SE.

404 errors are such a pain to deal with, as there are minor SE's out there that still try to call 3 year old pages (I know from experience). You're probably better off just storing the errors in a database, and viewing the log via web page. You can set up the page to delete the errors from the database when you're done too.

I also send myself text messages to my phone, but only if a 500 error occurs. They are so rare now that it doesn't bother me when I get one. I still log all my 404s and 500s to a database though, and delete them when I'm finished looking at them.

[larryhatch]

I'm a bit weak on the PHP stuff.

If I want to know who is calling up 404 pages or images,
I go straight to my ACCESS_LOG files. Bingo. Unambiguous.
If my host refused to provide Access files, I'd be looking for a new host. -Larry