G crawling and indexing a large site

My client has 60 million pages to get indexed in search engines

[MetricsGuru]

A client of mine is a parts supplier with 60,000,000 parts they'd like to be indexed in search engines, particularly G.

These pages would be generated dynamically with the part number in the title, description and keywords tag, along with part information.

The pages can be generated fairly easily, according the the developer, but getting them crawled and indexed is concerning me.

I've seen large sites with 700,000+ pages use a series of site maps to get it's pages indexed. But what do we do with a database of 60 million parts that my client wants indexed?

Any ideas would be appreciated. I hope this is the right catagory to post this question.

Thanks,
MetricsGuru

[cooldoug]

Google crawls links, just submit the main page, and all the links will ba added. I think:), That may have changed:)

[andrew_m]

Presumably, there is some sort of category structure behind that many products? Just make sure that the category structure is crawlable and does not use drop-downs/javascript to advance to next level (or if it does that there is an alternative route).

If these are genuine products the client sells you may also submit the whole thing to Froogle as a .csv dump.

[andrew_m]

Btw, don't expect to get all products into index if your home page PR is below 8 for that many products. You might want to group and expose only logical groups instead of individual products -- I bet they are all very similar and easily groupable with that total number.

[MetricsGuru]

Froogle seems like a good idea, but what about the 60 Million items part of it? Would Froogle want a feed with 60 million part numbers?

What do you think? It's the number that I'm worried about. Crawling that many pages would take months, at an average 2 pg per second crawl rate.

[andrew_m]

Ask froogle, I'm not qualified. And I doubt anybody here is. And think about grouping -- I doubt very much that each individual product really is that unique.