Welcome to WebmasterWorld Guest from

Forum Moderators: open

Message Too Old, No Replies

How to index only static pages not dynamic?



8:59 pm on Jul 13, 2005 (gmt 0)

10+ Year Member

We would like to employ an HTML catalog of a webstore which would mean having have 2 versions of each webpage ...dynamic(.php) and static(.html). Both pages will be exact copies of each other except that the links on the static webpages will point to .html instead of .php webpages.

What is the best way have ONLY our static pages indexed by the search engines and not the dynamic pages ...we don't not want to get duplicate content penalties.

1. To reduce server load when being crawled
2. To reduce server load from paid search engine listings
3. To reduce server load when visitors click on our listings in search engine results pages
4. To deliver the same content to both visitors & search engines

1. Cloaking?
2. Apache redirects?

Note that in our case I do not think cloaking by definition would cause any penality since the page delivered is almost identical except for the links?


3:11 am on Jul 15, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

You could do it without cloaking by displaying a <meta name="robots" content="noindex"> tag on the PHP version. That wouldn't help with bandwidth costs when the pages are spidered, though.


3:27 pm on Jul 15, 2005 (gmt 0)

10+ Year Member

Let me add some more information to help clarify some points:
By having the duplicate HTML pages listed in the search engines instead of the PHP & pointing our CPC URLs to the HTML pages instead of the PHP we can achieve:

1. Reduced server load(dedicated server) since the HTML pages will have NO php code to process.

2. The customers will not be browsing the duplicate PHP pages until the've added a product to their cart from an HTML product page which would would have the add-to-cart button submit to cart.php NOT cart.html ...thus the vast majority of our new customers would be able to browse through as many HTML pages as they like until they add a least 1 product to the cart. This should definitely allow for much more vistor traffic. At least that was our plan ...any better suggestions?

We plan on utilizing Zend Optimizer again to help speed up things on the server during the holidays ...made a tremendous difference last year!


10:39 am on Jul 27, 2005 (gmt 0)

10+ Year Member

Why not have two directories one for static pages and one for dynamic, then refuse entry to the dynamic directory via robots.txt?


1:16 pm on Jul 27, 2005 (gmt 0)

10+ Year Member

How reliable is employing a robots.txt file with regard to Google, Yahoo & MSN? 100%?

We definitely don't want both pages, static & dynamic, indexed by any search engine otherwise penalties could results since the html output would be the identical for both...


Featured Threads

Hot Threads This Week

Hot Threads This Month