Two domain names for same site, merge back into one

[AG4Life]

I have a website using example.com, but also has example.net that redirect to the site. example.net would do a 301 redirect to example.com.

After a server change a few months ago, the redirect failed and instead of redirecting to example.com, it would display pages from the website with the url example.net.

The redirect has been fixed, but it was a bit too late and now Google lists pages for both example.com and example.net in the index. When doing a site:example.net, it brings up a few hundred results. I made the fix about two months ago, and the listings for example.net still remain.

My question is, how would I go about trying to remove the listings for example.net, or should I simply wait until Google re-crawls those pages and find that they are now redirects?

Thanks.

[edited by: tedster at 4:38 pm (utc) on Oct. 18, 2009]
[edit reason] swutch to example.com - it can never be owned [/edit]

[tedster]

If your 301 redirect is working, there's no reason to be concerned - just be patient and let Google sort it out at their own pace.

[AG4Life]

Thanks.

[AG4Life]

Just an update on this, Google is still very slowly removing the indexed pages for the unwanted listings (on example.net).

I have a related, but a bit more serious problem, also to do with incorrect redirects leading to getting pages indexed that shouldn't have been. And this might even have led to a filter that the website is currently experiencing.

What happened was that the site had cached search results that would expire after some time, the links are like:

example.com/search.php?search_id=123456

Through bad SEO, these pages were not blocked in robots nor were set to noindex status, and using meta refresh instead of 301 redirects when they expire, this led to a lot of these pages to be indexed. And I mean a lot! So much so that the majority of pages indexed for the website happens to be these sort of completely useless links.

So the question is how to go about removing these links, preferably as quickly as possible? The way I'm currently doing it is:

* Add noindex meta tag to these pages so they would never get indexed again (should I use "noindex,follow"?)

and

* 301 redirect the expired pages and wait patiently for Google to re-crawl all the links. I'm currently redirecting to a page containing the search box, which isn't linked to from elsewhere (since the search is built-in to the interface of the website, and there's no "advanced search" page) - is this a bad thing (to 301 redirect to a page that's not linked to anywhere)?

Thanks.

[AG4Life]

Just an update to say that the filter/penalty has been lifted just today. Yesterday, the number of results returned by site:example.net was 600+, today it was only 300. Whether that was the difference or not, I'm not sure.

As to what I did in regards to the cached search results, I did add the noindex,follow tag, plus 301 redirected the expired pages. Then waited patiently for Google to de-index everything, and 90% of these cached pages are now no longer indexed (although this occurred two weeks ago and the filter was not lifted then).

It was probably a combination of pages being indexed for example.net, and the cached search result pages that caused the filter in the first place.

So the moral of the story: be very careful when using redirects and multiple domain names pointing to the same site. Avoid double redirects, and "noindex,follow" (or "noindex,nofollow") anything that's of no value.

Note that it was very hard to find the pages that shouldn't have been indexed through Google's "site" command. Doing a "site:example.com" doesn't actually include these useless pages for me. At one point "site:example.com" returned 3,000 results, while "site:example.com/search.php" returned 8,000 results.

[tedster]

Also note that once a 301 redirect is served, googlebot cannot even see the content for the previous URL - so the robots meta tag can't be read.

[AG4Life]

That's right, the robots meta tag is to prevent new pages from being indexed, which is also an important step in this whole process, as for a short while, these temporary cached results pages did display content (and only became a 301 redirect after they expired).

I also made the mistake early on of using robots.txt to block the pages that I didn't want to index, believing at first that not that many of these pages had been indexed. But many of them had been, and blocking them via robots.txt only prevented Google from re-crawling them to detect the 301's and remove them.

[maximillianos]

Probably no longer necessary, but you can remove pages from G's index using their Webmaster tools.

[Hoople]

I went thru this in October 2009 and have seen it take a few weeks to months for all of .net the pages to drop in Google.

Bing still is quite a bit slower, it keeps looking for pages gone in 2007!

[AG4Life]

Looks like I may have jumped the gun. The filter or whatever it is came back 12 days later. Looks like it could have been Caffeine related, since the filter originally happened about two weeks after Google announced Caffeine, and the weird Caffeine related movements around Feb 7/8 is when the filter was lifted.

The symptoms of the filter are:

* For top 20 popular keywords (via Google Webmaster Tools), where they used to rank on the first page, are way back at 900 something.

* Some keywords, for which I'm top, are still top, although these are not as competitive (but still easily beating the likes of big websites like Wikipedia in the SERPs).

* For these affected keywords, even typing "keyword + example.com" doesn't get the pages to 1st spot, the first couple of spots are usually pages from the same site that aren't affected, or worse, from another totally unrelated website. Only doing "keyword + site:example.com" brings the affected pages to the top.

* New content gets indexed but never gets past page 4 or 5, despite the content being very unique (niche website). Pages that link to the content are placed higher in the SERPs than the actual content, sometimes including the pages on the affected domain

[Ma2T]

In Google webmaster tools there is an option for "Change of address".

If you add both domains, you can tell google that one domain has now changed to the other. It will kind of link them together, knowing one points to the other.

Direct to the horses mouth might be good.

[AG4Life]

Most of the site is ranking again, although there are still some missing pages compared to the last time it was back. Don't know how long it will last, but I suspect whatever it is, has to do with the transition to Caffeine.