Best way to remove pages from Google

[CWebguy]

I have a site that previously had some pages like

index.php?page=links etc. etc. which are now rewritten differently.

What's the best way to remove these from Google's index as they are both now showing up (both versions of the same page). (I can't 301 redirect them due to the way it is set up, or I don't know enough rewrites to perform the operation). I just want to remove the old pages, (which aren't even linked to anymore), and keep the new.

Thanks.

[phranque]

Urgent removal request - Webmasters/Site owners Help [google.com]

[CWebguy]

I've tried the removal process in the past, and they have said that it wouldn't be processed since the page still existed (or was linked to, even though it wasn't) or something. I can try again though, thanks.

Note: Reading that I have seen that the page must give a 404 in order to be removed. How can I do this? Can I use a mod_rewrite with [G] (gone) flag?

Thanks.

[edited by: CWebguy at 4:03 pm (utc) on Mar. 8, 2009]

[Gandhalf]

when you use the url removal tool, you need to also add the url's in your robots.txt
It's very boring when you have lots of them but it works

[Gandhalf]

like:
disallow: /thispage.php?index=3

[CWebguy]

ok thanks, didn't know robots could take individual pages, but I know that Google is more flexible with it's robots syntax, thanks. right now I have been using mod rewrites with gone flags, don't know if this is a correct solution but seems like it should be.

Thanks

[edited by: CWebguy at 5:56 am (utc) on Mar. 9, 2009]

[phranque]

when you use the url removal tool, you need to also add the url's in your robots.txt

not always - if you add that url to robots.txt then the bot will never see the 404/410.
if it is truly gone, you should say it's Not Found/Gone, not Disallowed.

from the above referenced google support document:

To remove a page or image, you must do one of the following:
* Ensure requests for the page return an HTTP status code of either 404 or 410
OR
* Ensure that the pages you want to remove have been blocked using a robots.txt file
OR
* Ensure that the pages you want to remove have been blocked using a meta noindex tag.

[CWebguy]

On one site I 404'd it plus Robots. On another just did the Robots so will see how Google reacts. Thanks.

[phranque]

as far as googlebot is concerned those are the same signal - disallowed.

[g1smd]

Best way is to use a 404 or 410 status code, but only for pages that do not exist at all. For pages that have simply moved to a new location, use a 301 redirect.

Especially in the case of a redirect do not force Google to drop the old URLs, because while they still show in the SERPs they will still deliver visitors to your site, and your redirect will get their eyeballs on the correct content. That is, by forcing removal you might be removing a source of traffic to your site before Google has fully indexed and evaluated/scored/ranked the new URL for that content.

[CWebguy]

Well, good news. Pages are gone after only about 12 hours. Google was quick on this one using Webmaster tools ;)

[g1smd]

Early removal of the old URL can lead to a loss of traffic if the new URL hasn't been fully evaluated/scored/ranked - so quick removal isn't always good news.