homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Gold Sponsor 2015!
Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

SiteMaps + Large site =?
Indexing large sites with sitemaps

10+ Year Member

Msg#: 3089208 posted 9:04 pm on Sep 19, 2006 (gmt 0)

Hi all,

I have several very large sites (200,000 to 2,000,000 pages of unique-ish content). With my largest one (the 2 million pager) I've deliberately not linked to all pages out of fear that Google will consider the site too big and drop it. Is that a rational fear? And if so, would having a Google Sitemap for the whole thing be potentially detrimental? I have a decent amount of referrers to lose out of this.



WebmasterWorld Senior Member whitey us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

Msg#: 3089208 posted 10:06 pm on Sep 19, 2006 (gmt 0)

According to Matt Cutts

Launching sites with millions of pages [video.google.com...]

the release of more than 5,000 pages a week will raise a flag and likely not index.

If it does, at this rate, it will take 400 weeks to release a 2M page site.

My guess is that if the site is of exceptional value, Google might take notice and release it, if you contact them, but you'll need to be something like Wikipedia to get away with it.

[edited by: Whitey at 10:07 pm (utc) on Sep. 19, 2006]


10+ Year Member

Msg#: 3089208 posted 7:06 am on Oct 9, 2006 (gmt 0)

Excellent link, thank you.

My site is 2 years old though. Matt Cutts is talking about when you launch. How bout a whole lot of content that GOogle wouldn't have found before?


5+ Year Member

Msg#: 3089208 posted 8:50 am on Oct 9, 2006 (gmt 0)

I opened a members only section to the general public and launched 16.000 'pages' in one week as the spider finally got access to hose as well. They all got indexed in one week time.

As far as the sitemap, I submitted pages that have all the content links added and that seems to have done the job pretty well. Mind you, these link pages are accessible for general users as well.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved