Forum Moderators: phranque

Message Too Old, No Replies

mod rewritten pages not being indexed

Mod rewritten pages not being index by search engines, have I coded this in

         

frankstuner

12:27 am on Dec 10, 2011 (gmt 0)

10+ Year Member



I've recently added some mod rewrite rules to my site, and they function exactly as I need, but the pages aren't being found by google or bing after multiple crawls. I also used a sitemap generator and it couldn't find my mod rewritten pages either. I've got put in the correct title and meta tags in the page.

I've done some searching about what I could be doing wrong but haven't come up with anything that seems to cover what I'm experiencing.

I used the fetch tool in google webmaster tools it said it hadn't found the page but it did show up the source code for the page, the correct page...

Any idea of what I'm missing here?

Cheers,
Frank

g1smd

1:23 am on Dec 10, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Google should easily find all of your new URLs, you should be linking to them from within the site.

If Google is continuing to ask for the old URLs, and they will do so even when you no longer link to them, you should also add another RewriteRule to redirect requests for the old URLs to the new URLs.

You should no longer link to the old URLs from within the site itself. Use Xenu LinkSleuth to be sure you're not doing so.

Use the Live HTTP Headers extension for Firefox to be sure the correct '200 OK' HTTP response code is being returned for the new URLs.

lucy24

2:13 am on Dec 10, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Painfully obvious question: When you say "rewritten" do you mean "redirected"? If the pages are only being rewritten, you don't want g###-- or anyone else-- to find the real address. That's the whole point of a rewrite.

Even after they've located the new address-- assuming that we are talking about a redirect-- they will periodically keep asking for the old address until {the cows come home | pigs fly | hell freezes over} Not sure which. Especially if :: cough, cough :: one of your own pages still has an overlooked link to the old URL, so it's continuously being reinforced.

MikeNoLastName

8:23 pm on Dec 12, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I don't think it really matters, even if you DO get rid of all the internal old links these days, if someone else has linked them (like your old home page being 301 redirected to a new domain home page) they're never going to stop being crawled. No way you can find and contact 30,000 websites (in our case) and ask them to change their links.
I'm trying to get rid of some right now which were 301'd over a year ago and were totally out of the index a year ago, and now are back indexed as duplicates of our new home page! And crawled twice a day. Just can't get rid of them!
Was thinking of just replacing them with a "click here to go to our new site" and a a single out-link to the new domain. At least we'd get most of the the old PR and no dupe penalty