Removing URLs that are 301ed

Google won't let you anymore

[steveb]

The old URL removal tool [services.google.com:8882...] has unfortunately been taken offline, probably due to a thread here. The URL now leads to the feeble webmaster tool ability to remove some URLs... but not URLs being 301ed to a new location.

This user-unfriendly behavior makes no sense as webmasters want visitors to end up on the right page all the time, not have 404 errors waiting for Google to find time to clean up after themselves.

Has anyone found a work around? I'm not going to say screw you to a couple hundred visitors even for a day waiting for Google to do what used to take it less than five seconds.

[g1smd]

Google shows URLs that are now 301 redirects in their index as Supplemental Results for a year before they clean them up.

You can't change that, and you should not worry about it. The old URLs still deliver visitors to your site, and your redirect takes them to the correct page.

[steveb]

That wasn't the question, and of course you should worry about URLs being listed that don't exist unless you are in the business of confusing people, and more important if you don't want to get a ranking for a term mistakenly applied to a supplemental rather than a healthy page.

[tedster]

Within the webmaster tools you have three methods:

To remove content from the Google index, do one of the following:
1. Ensure requests for the page return an HTTP status code of either 404 or 410.
2. Block the page using a robots.txt file.
3. Block the page using a meta noindex tag.

If your url returns a 301, then you cannot use #1 or #3 -- but #2 will work.

[steveb]

#2 leads to the pages not being crawled, but it should leave it in the index.

[Bewenched]

Yup .. it has to go to a page that sends a 404. which really sux since any other links to that page from organic linking will also get a 404.

[tedster]

I've recently had urls removed by using #2. Beneath the section I quoted above, it says:

If you need to expedite your content removal, make sure you have met one of the requirements listed above, and then select the New Removal Request button below to use this automated tool.

Google does not explain this very clearly in the process of clicking through to that page. But in the two recent cases where I used it successfully, the content had to stay available for direct access by a few people -- however, it couldn't be allowed to show in any search results. The GWT url removal process did work, and in under 3 days. After one more crawl - the party was over.

[steveb]

As stated above, the issue is having a 301 on a page. It appears you can still remove pages that are online, but you can't remove URLs that are currently 301ed unless you take off the 301 and wait three to five days.

Previously all you had to do to remove a page was take a 301 off for five seconds. Now it is five days, which is absurd if you are redirecting multiple pages via the same htaccess command.

[tedster]

I appreciate the frustration, steveb. I wasn't aware that googlebot needed to spider the url in the case of a robots.txt removal. That doesn't make sense to me -- all they need to validate the removal request should be robots.txt.

[giggle]

We are in a similar process handling the 301's via ASP, replacing the city level page with a 301.

I was wondering this: Currently the site map links to countries and then, on the next level site map, to locations. We're changing the location (city) level pages to point back to the WWW. Should I immediatly remove the links from the site map that points to the 301'd location pages or should I wait for a period of time so that Google crawls the site map to find the 301'd pages then remove links to those pages from the site map?

Thanks

Mick