Forum Moderators: phranque

Message Too Old, No Replies

How to tell Googlebot 404 instead of 302?

Custom Error says 'moved' -- need 'not found'

         

Pfui

12:09 am on Jun 27, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Googlebot keeps looking for non-existent pages -- well, the pages exist, but without all this junk appended:

06/26 05:45:30 /dir1/example1.html%3E%3Cimg%20src= 302 -
06/26 05:45:38 /dir1/example2.html%3E%3Cimg%20src= 302 -

(The junky parts aren't my code so either someone's copy-pasted and/or messed up a link somewhere.)

Currently everyone bumping into a 'normal' 404 File Not Found gets 302'd to a Custom Error [httpd.apache.org] page -- including Googlebot.

ErrorDocument 404 http://www.example.com/404.html

So either I'm doing something wrong (likely:) or I'm hoping I can just tell Googlebot to, um, ignore 302s?

SetEnvIf Request_URI "%3E%3Cimg%20src=$" ignore
PassEnv from env=ignore

RedirectMatch 301 Googlebot .....?

BrowserMatch Googlebot!302 .....? (there's a space before the exclamation mark)

(Yeah, I know. None of those make any sense but SetEnv [httpd.apache.org], etc., is all I could come up with that made any sense to me, sort of:)

TIA for your help!

jdMorgan

12:20 am on Jun 27, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



This line is incorrect, and is the source of your (very common) problem:

ErrorDocument 404 http://www.example.com/404.html

As specified in the ErrorDocument description [httpd.apache.org], if a canonical URL is given, then the server response will be a 302-Found, since that is the only way to 'reach' an possibly-external domain in a manner which does not imply permanence. To correct this problem, simply use

ErrorDocument 404 /404.html

Jim

Pfui

2:49 am on Jun 28, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



This just in --

crawl-66-249-65-194.googlebot.com
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
06/27 19:22:50 /dir1/example1.html%3E%3Cimg%20src= [b]404[/b] -

Perfection. Thank you!