Site not indexed after 4 months.

[Scott00]

My site was rebuilt and went from php to htm. The new site was loaded before the new year and the new pages have yet to show up on search engines. But the old (php) pages are still there. If I go to yahoo, search my site and click on "more from this site" all of my php pages (and some very old html pages) come up. But no htm pages...
When I try to check the cache of the new site it says "error, requested url was not found on this server". This happens for all search engines.
As far as I can tell the new "htm" site has never been indexed. At the beginning everyone told me to be patient but it's been over 4 months...
I did have page ranking before, I do have backwards links and a sitemap. Basically, it is the exact same site that the php was, it was just changed to htm.

[BarHopper]

Basically you have flagged every indexed paged from your old site and conflicted it with your new site. You shouldn't have done this. This is pretty much starting you out from scratch and has potential to get you nocked for dupe names and or content that has previously been cached. And 4 months isnt that long it can take anywhere from 4-9 months with google depending on the structure and plan you have gone with your site.

-Bar

[Demaestro]

Do you have all the same base URLs just with a .htm rather then .php?

If so that is very strange, and even more reveling is the fact that you are having this problem with all the major SE not just the main 1. Do you have any logs showing that some of the big SE bots have pai you a visit?

I know this doesn't help but it should be noted that if you only changed the file extentions and this is what happened, that this would be an excellent argument to hide file extentions for your websites so that if you do switch the backend technology you can keep the same site and no-one has to know all your .asp pages became .php

[Scott00]

About 50% of the page names are the same as the old site. But the rest are new names.
In the microsecond between clicking on "cached website of page" and the error page I can see that google did crawl the site on March 2nd.
It sounds like I've done something that may be misconstrued as unethical... Is that the case?
I switched from php to htm because that was what the new site designer wanted to do.
Could it be something in the code?

[Demaestro]

I doubt it could be the code, unless the pages are very dynamic and require a query string to be passed in to get unique content.

Do you have a Google Sitempas account?

If so you should check some of the reports. See if there were any errors reported by the bot. 4 months isn't that long but you should have been at least cached by now.

If you don't have a google sitemap account I suggest setting one up and see if the bot can tell you anything. Becuase this is happening for more then 1 SE I would say that it has to be something obvious. It is rare that the SE bots all act the same unless it is for some obvious reason. It could happen but I doubt that the big SE would all have the same penilty for some obscure reason.

[Scott00]

I don't have a sitemap account but will definitely look at getting one!
I just talked to my web designer who had some thoughts. She left the old pages in the www file and thought that perhaps the engines were getting confused? She has just removed the old files.
Also had a problem where the entry page was the index.htm instead of just website.com. When the old index.php was removed then the entry page was just the website name.
It seems like the crawlers may have been getting confused because the old files were still there.
Does that sound reasonable? And if that's the case am I going to have to wait another 4-9 months for this to get straightened out?

[Demaestro]

I would say that could be the culprit. If you have a different set of papges at the www subdomain then you do at the no www TLD then that really could be the issue.

The worst is you make those changes then you have to wait to see if it worked, how long yuou should wait before trying another solution is anyones guess but it sounds like you could have a few more months on your hands before you are back in the SERPs.

[Scott00]

Thanks for the help!
It looks like it's either a wait and see game or a hire a very expensive person to look at the site kind of thing...

[caveman]

Scott00, I'm still not clear after reading this thread...

Sounds like you had old pages with php extensions, added all new identical pages with htm extensions, and did not 301 or remove the old pages. If you left the old pages in place then the spiders would still be indexing them and that is likely the core of your problem. You had an entire duplicate content set of pages.

Here's what should have been done:

301 all old php pages to their new htm counterparts. Mod_rewrite works well for this. (Note: If you've not done this yet, and just took down the php pages, you should still immediately implement 301's.)

Make sure that all internal links on the new htm pages reflect the new file extensions (so you don't have links pointing to php pages). Can't tell you how many times I've seen this mistake.

Try to make sure that all internal links follow the same format, so you don't risk confusing the SE's. This is particularly important for the homepage, but really needs to be done site wide. That means all internal links to the homepage, for example, are in the form "examplesite.com/" or whichever form you choose, and that some of the links to the homepage don't look like "examplesite.com/index.htm" and others like "examplesite.com".

For any external links pointing to your site that utilized the old file extensions, try your best to get them over time to change their links.

Assume that in G's case your site may drop from the SERP's for 3-12 months but that in the other SE's, the changes would appear after serveral weeks in the best case (MSN first) and several months in the worst case, unless the 301's were improperly handled, or Y was taking its time between updates.

Make sense?

[Scott00]

Caveman

I think you got it... Site was php, new web designer wanted htm. Alot of the page names are the same but alot aren't. Content and design is all different.
Originally there was an auto re-direct from the old pages to the new. I'm not sure if it was a 301, I'll check. Now it goes to a 404 page.

Everything else about the links makes sense and thank you for your help!