Forum Moderators: phranque

Message Too Old, No Replies

Getting over 1 Million product pages indexed

Need advice on how best to launch

         

zorde

5:48 am on Oct 22, 2009 (gmt 0)

10+ Year Member



I am about to launch a large site with over a million products, they are nicely categorized and individually described - so every page is unique content.

My concern is how many of those product pages Google and other SE's will actually index.

If it is limited, I want to ensure the SE pays crawls my priority product pages.

Anyone have any experience with getting sites indexed that have massive amounts of pages?

I am thinking I should launch with say 10,000 product pages and the slowly drip feed a few hundred pages a day? Would that work?

phranque

6:39 am on Oct 22, 2009 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



i would recommend:
- good internal navigation
- relevant inbound links
- setting up your server to respond appropriately to "If-modified-since" request headers
- proper use of the "Last-Modified" and "ETag" response headers
- starting with a limited (to your priority pages) xml sitemap [google.com] with careful attention paid to the <lastmod>, <changefereq> and <priority> tags.

zorde

7:04 am on Oct 22, 2009 (gmt 0)

10+ Year Member



very good points,

Do you think it is feasible to have >1M pages?

phranque

7:52 am on Oct 22, 2009 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



feasible?
you mean like (poof!) there's a million(+) unique pages of editorial content?
that's 100 pages/hour for 417 days.
i can believe that with a template and a feed or a database you can come up with a million unique combinations.
that's not quite the same as writing about a million unique products.
there is some overlap - it depends on your definitions.