Welcome to WebmasterWorld Guest from 23.22.220.37

Message Too Old, No Replies

Will the first time google crawls my site matter more than other times?

     
8:48 pm on Aug 29, 2010 (gmt 0)

Senior Member

WebmasterWorld Senior Member planet13 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:June 16, 2010
posts:3796
votes: 28


Hi there, Everyone:

I have a new e commerce site in development that will EVENTUALLY have about 12 different categories and about 180 products.

Should I block the site with robots.txt until ALL the products and categories are up in place?

Or is it ok to start letting search engines index the site if I only have two categories and maybe 15 products up, and gradually add categories and products as they become available? (there won't be any broken links or pages with minimal text while doing this).

In short, does the first crawl / indexing of a new web site have any more significance than subsequent crawling and indexing?
3:15 am on Aug 30, 2010 (gmt 0)

Preferred Member

5+ Year Member

joined:July 24, 2006
posts:386
votes: 0


I've done it both ways and have not really noticed any real difference as far as ranking. One thing is on the sites that I have blocked and then opened up with significant content is that it takes upwards of a month for google to get through the entire site. based on that it might be best if you are putting your best products in first to go ahead and let google at it so you can get those products indexed as soon as possible.
3:27 am on Aug 30, 2010 (gmt 0)

Senior Member

WebmasterWorld Senior Member planet13 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:June 16, 2010
posts:3796
votes: 28


One thing is on the sites that I have blocked and then opened up with significant content is that it takes upwards of a month for google to get through the entire site.


Yeah, now that you mention it, I remember that happening to another site I had done a while back...
6:12 am on Aug 30, 2010 (gmt 0)

New User

5+ Year Member

joined:Jan 29, 2010
posts:24
votes: 0


I do not have any hard facts to back my opinion but it's unlikely that Google would like to "profile" a site and keep grudges based on its first crawl.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members