Page is a not externally linkable
internetheaven - 1:22 pm on Jan 14, 2007 (gmt 0)
Really? We recently allowed Google to crawl one of our article archives (we were previously afraid of scrapers). There must be over 200,000 pages in there. Will I be penalised for allowing them access to these pages all at once? Should I be spoon-feeding the Google bot-children or even bottle-feeding them? Big news sites add hundreds if not thousands of articles a day. Big software/shareware sites add hundreds if not thousands of pages a day. Forums add hundreds of pages a day. There are many, many legitimate reasons why some sites grow really, really fast. They will all be penalised?
78,000 new pages in 21 days will trigger an alarm.