I have a new e commerce site in development that will EVENTUALLY have about 12 different categories and about 180 products.
Should I block the site with robots.txt until ALL the products and categories are up in place?
Or is it ok to start letting search engines index the site if I only have two categories and maybe 15 products up, and gradually add categories and products as they become available? (there won't be any broken links or pages with minimal text while doing this).
In short, does the first crawl / indexing of a new web site have any more significance than subsequent crawling and indexing?
I've done it both ways and have not really noticed any real difference as far as ranking. One thing is on the sites that I have blocked and then opened up with significant content is that it takes upwards of a month for google to get through the entire site. based on that it might be best if you are putting your best products in first to go ahead and let google at it so you can get those products indexed as soon as possible.