Error 406 with submitted URLs

Forum Moderators: open

Message Too Old, No Replies

Error 406 with submitted URLs

html scripted php3 pages using ForceType apache directive

dax

4:58 pm on Apr 25, 2001 (gmt 0)

We are delivering our php3 content using the ForceType apache directive with (apparently) no problems to normal browsers. However we have discovered that search engine crawlers don't like the content and return an Error 406 Not Suitable message. Something to do with content negotiation?

Example URL:
[laterooms.com...]

When submitted to Altavista at :
[add-url.altavista.com...]

we get

You have entered the following URL:
[laterooms.com...]
We tried to fetch this page but received an Error 406. The usual causes of an Error 406 are a mistyped URL or the page could not be retrieved from the Web site you entered. This page will be removed from the AltaVista index.

Any ideas??

Regards, Paul Walsh

Mike_Mackin

5:13 pm on Apr 25, 2001 (gmt 0)

Welcome to WebmasterWorld dax

This subject came up yesterday.
[webmasterworld.com...]

dax

5:19 pm on Apr 25, 2001 (gmt 0)

Those Error 404s are just outages - this is definately something else .

Error 406 means the server has a page which does not match the requested content headers (or something like that). Our php3 content works fine...

Paul

jeremy goodrich

5:24 pm on Apr 25, 2001 (gmt 0)

This is what I get when I send an HTTP request to your page listed as an Example URL:

HTTP/1.1 404 Not Found
Date: Wed, 25 Apr 2001 17:21:36 GMT
Server: Apache/1.3.12 (Unix) PHP/3.0.16 mod_ssl/2.6.2 OpenSSL/0.9.5a
Connection: close
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<HTML><HEAD>
<TITLE>404 Not Found</TITLE>
</HEAD><BODY>
<H1>Not Found</H1>
The requested URL /en/country1.html was not found on this server.<P>
<HR>
<ADDRESS>Apache/1.3.12 Server at wim.nation-net.com Port 80</ADDRESS>
</BODY></HTML>

So, their spider is going to have the same difficulty as me. What you'll need to do, my guess, is reconfigure your htaccess file...I don't know apache, though.

littleman

5:26 pm on Apr 25, 2001 (gmt 0)

[cgi-fun.hypermart.net...]
I am getting a 302 header form your page, not a 406.

dax

5:31 pm on Apr 25, 2001 (gmt 0)

OK that example is even more complicated - it also does a redirect (302)

Try this: (it gives 200 OK at hypermart but 406 in Altavista)

[uk.laterooms.com...]

Paul

littleman

5:40 pm on Apr 25, 2001 (gmt 0)

yeah you are right, very odd. Are you doing any type of user agent or IP detection?

dax

5:46 pm on Apr 25, 2001 (gmt 0)

No nothing like that. BUT ...

This could also explain why we can't get the same html(php3) pages into Google - all the php3 pages in the site are there but no sign of any of the handler generated ones ...

All I can guess is that the AV/Google agents are requested something like a charset or something which a normal html page would satisfy but which Apache is forgetting to add on to the ForceType handler
versions.

Paul

dax

5:53 pm on Apr 25, 2001 (gmt 0)

JEREMY - did you use a HTTP 1.1 or 1.0 client when you got that error 404?

(we use 1.1 - virtual hosts)

Paul

engine

5:56 pm on Apr 25, 2001 (gmt 0)

Now, I noticed this too, yesterday, I put it down to a bad day, but, if others are having problems too...

I won't be revisting the submissions until Friday now, but will watch this thread.

jeremy goodrich

6:36 pm on Apr 25, 2001 (gmt 0)

You're second one is here:

HTTP/1.1 404 Not Found
Date: Wed, 25 Apr 2001 18:34:45 GMT
Server: Apache/1.3.12 (Unix) PHP/3.0.16 mod_ssl/2.6.2 OpenSSL/0.9.5a
Connection: close
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<HTML><HEAD>
<TITLE>404 Not Found</TITLE>
</HEAD><BODY>
<H1>Not Found</H1>
The requested URL /en/region106.html was not found on this server.<P>
<HR>
<ADDRESS>Apache/1.3.12 Server at wim.nation-net.com Port 80</ADDRESS>
</BODY></HTML>

Notice in the top, the header information it says http 1.1.

littleman

6:39 pm on Apr 25, 2001 (gmt 0)

200 OK
Connection: close
Date: Wed, 25 Apr 2001 18:37:57 GMT
Server: Apache/1.3.12 (Unix) PHP/3.0.16 mod_ssl/2.6.2 OpenSSL/0.9.5a
Content-Type: text/html
Client-Date: Wed, 25 Apr 2001 18:38:48 GMT
Client-Peer: 194.200.209.11:80
Link: ; rel="stylesheet"
Title: LateRooms
X-Powered-By: PHP/3.0.16

Is what I get for the same url using an lwp script.

jeremy goodrich

7:31 pm on Apr 25, 2001 (gmt 0)

My first try at this, I must admit, trying to help in this thread is helping me a lot too.

Here is what I got using a simple "get" perl client:

from the command line:
perl wget.pl www.uk.laterooms.com /en/region106.html

the resulting output:

HTTP/1.1 404 Not Found
Date: Wed, 25 Apr 2001 19:29:12 GMT
Server: Apache/1.3.12 (Unix) PHP/3.0.16 mod_ssl/2.6.2 OpenSSL/0.9.5a
Connection: close
Content-Type: text/html; charset=iso-8859-1

dax

9:38 am on Apr 26, 2001 (gmt 0)

The server uses virtual hosts so any 1.1 request must send the
Host: www.uk.laterooms.com header together with the GET . Not sure if the perl lib does this - Try these 2 requests using telnet:

telnet www.uk.lateroms.com 80
>GET /en/region106.html HTTP/1.1

404 Not Found etc.

telnet www.uk.lateroms.com 80
>GET /en/region106.html HTTP/1.1
>Host: www.uk.laterooms.com

200 OK etc.

dax

1:12 pm on Apr 26, 2001 (gmt 0)

PROBLEM SOLVED !!!

Do not use the ForceType x-httpd-php3 directive to alias a directory to a script - I have found that the following Accept headers are rejected with an Error 406
Mime Type:
Only
Accept */*
and
Accept application/x-httpd.php3 works
Many browsers send this but not custom bots
Accept text/html and text/* DO NOT !!

Also only Accept-Language: en works - no others!

Instead I am now using:
Alias /en/ /htdocs/laterooms/en.php3/
etc.

within the VirtualHost container

Seems to be working fine - no more Error 406s from Scooter and hopefully Googlebot will be happy too.

Paul