header("Location:")

With apostrophes and ampersands in the URL

[barns101]

I'm redirecting a lot of old URLs to new "static" equivalents and all is fine using IE and FireFox but I've got a feeling that Googlebot (and possibly other spiders) and Opera are encountering problems.

I've searched to see if Googlebot and Opera have specific problems but can't find anything conclusive. Can anyone shed some light?

The PHP code is like this:

<?php
// Gets new URL from database depending on $_GET["id"]
header("HTTP/1.1 301 Moved Permanently");
header("Location: http://www.example.com/$area/$name/");
exit();
?>

And so the browser would receive something like this:

header("HTTP/1.1 301 Moved Permanently");
header("Location: http://www.example.com/city-centre/old-queen's-head/");

Everything works fine in IE and Firefox, but checking my log of 404s I see that Googlebot requested http://www.example.com/city-centre/old-queen (the URL stops at the apostrophe) and an Opera user requested http://www.example.com/city-centre/old-queen%27s-head/ (the apostrophe is replaced by its encoded equivalent).

I can update my actual links to use the HTML entity of the apostrophe, but will Google and other be able to follow the dynamic redirects?

[Habtom]

I wonder if they would. I have had similar problems on one of my sites. After I changed it all, my page ranks have gone down. Plus, Google indexes only some part of the site, and not well.

Habtom

[eeek]

Just encode with ' with its %xx version.

[Habtom]

Hi,

You might want to look at urlencode [php.net]

Habtom

[barns101]

Thank you for your replies.

I think that the HTML links are being followed (and I will also encode & and ' as well). Its just the header("Location:") that is breaking.

For the header redirects I could URL encode and then decode the string, but would Google's SERPs show http://www.example.com/old-queen's-head/ or http://www.example.com/old-queen%27s-head/?

[jatar_k]

I would not put the ' and & in the urls as an ampersand has another meaning all together when it appears in a url

I would strip them out and make them something else, probably

' => nothing, remove all together
& => replace with the word and

no reason to make things complex for indexing, always make it as simple as possible. If you are not sure it will be followed or you see some issues, then simplify ;)

[barns101]

I completely agree and ideally I would not have them in the URL at all.

However, the first "variable" is an area of my city and the second is the name of a pub, as it appears in my database. All the URLs are rewritten to a single PHP script that takes the area and pub name provided in the $REQUEST_URI and displays the details of that database record. This has replaced my previous index.php?id=123 format.

How would you suggest doing it without using the apostrophes and ampersands, bearing in mind that they appear in names that will be searched for? I don't think that I could replace ampersands for the word "and", because that may cause the script to find a match with another existing record in the database.

Any help / suggestions appreciated. I may have missed an obvious way of accomplishing this. :)

[jatar_k]

well

you could store the pagename as a seperate field
or
you could str_replace the chars when you gen the page

[barns101]

The number of URLs that will contain special characters is minimal and Google does seem to be able to follow the new HTML links and has indexed the pages without any problems, so it looks like this will only be an issue whilst people (i.e. Opera users) are accessing the old URLs and getting the 301 redirect.