AOL messing with my URLs

& becomes &

[uplate]

Hi, I'm getting referrals from Google with no problem, but the AOL serps are messing up my URLs as follows:

www.mysite.com/mypage.php?op=xyz&id=6 (real URL)
www.mysite.com/mypage.php?op=xyz&id=6 (AOL version)

Anybody else had this problem?

[olias]

Yeah I'm getting that problem too it is most frustrating.
I have directories that are of the format /this&that/. Must get round to making sure those people aren't getting 404 errors, thanks for the reminder :)

[crash]

It's just like when a blank space get's converted to %20 (i think that's it.. need coffee still...)

it's the raw code for the char

example (google searches)

http://www.google.com/search?hl=en&lr=&ie=ISO-8859-1&q=somesite

vs

http://www.google.com/search?hl=en&lr=&ie=ISO-8859-1&q=somesite

(note in the 2nd url I changed the last amp to & from & you get the same results

[olias]

When you click on a link in an aol serp it is passed through a redirect (search.aol.com/redir.adp) which has the urlencoded format of the url that is is supposed to redirect to.

For some strange reason though it converts '&' to '&' which is the html safe version of '&' and also encodes the '&' to '%26'. Okay I have no problem with them using & whenever it needs displaying in the html, but it shouldn't be in the url.

The practical upshot is that i get requests for www.example.com/This&That/Widgets/ which of course doesn't exist.

[gsx]

Because & is a symbol that is used for seperating variables. The standard states that you should use & in URLs - AOL have it correct - you have it wrong (strictly speaking).

The '&' character should not be used in filenames.

Incedently, if you used & (the correct format) for a variable separator in a URL, could this cause a problem with any browsers?

[olias]

Yeah I know I shouldn't have used & in a file or directory name, but hopefully I will learn from that mistake, it has just never been a problem before.

And while AOL have it (strictly speaking) correct, I am of the opinion that it is more by accident than design, since I have no problems from aol.co.uk and aol.com.au

[ciml]

Most Web content is of the SGML HTML varieties (and I'm a luddite) so I'll quote from the HTML4 spec's here.

A href is defined as %URI [w3.org] and URI is defined as CDATA [w3.org]. In CDATA, user agents should Replace character entities with characters [w3.org]. When I checked (a couple of years ago), WannaBe, Opera 3.5 (Windows) and Amaya 2.1 (Windows) were broken in this regard but everything else that was tried (Mosaic, WebTV Viewer, Linkbot, iCab, Nav and IE) de-entified correctly. I seem to remember that Opera fixed the problem since?

Anyway, AOLSearch would be correct to link to a URL of x&y using x&y in the href attribute, but this isn't what we're seeing. Because the semicolon is URI encoded, AOL's redirect script is sent xamp%3by by the browser (which shouldn't and doesn't de-entify &amp%3b). The redirect script then issues an HTTP status 302 redirect to x&y which is wrong because this is an HTTP header and not an HTML attribute value. It also sends an HTTP body with x&y as the link which is correct because the browser will de-entify.

It should be simple to fix.