Forum Moderators: Robert Charlton & goodroi
I am monitoring a new site that was launched. The new site has quality links pointing to it - googlebot has been by and picked up some pages and cached them. Deepbot has not been by yet for a big feed. It has been about 10 days since Google first found the site. I would have expected deepbot to come by by now and go on a tear.
I would appreciate it if anyone would share their experience on the speed with which googlebot spiders new sites, particularily large sites with many (1000+) pages.
A Google sitemap has not yet been submitted. This is an option for the future, but for now, we want to see how google would find the site on its own.
The site is static html and its layout presents no problem for Google to find all pages.
Thanks.
Is it still true that deepbot IP addresses start with 209 and fresbot IP address start in the 60's?
For example:
Deepbot: 209.X.X.X
Freshbot: 64.X.X.X
Is deepbot coming out once a month or so or is it more frequent than that?
i set one site up a few weeks ago - the home page was listed in google within 6 hours - first clickthroughs within 3 days - whole site (approx 150 pages) listed in google within 5 days
other sites have taken anything from 1 week to 6 weeks to get listed
every time i set up a site, i add a link to it from a single links page on one of my other (rubbish) sites - proper link building and submissions to DMOZ / yahoo etc follow later - i do nothing else
MSN is also quick to list home pages plus anything linked from the home page, but is slower to list the entire site - i generally get better traffic from MSN in the first couple of months than i get from google
i have no idea if the google bots are fresh bots, deep bots, or other ............ and to be honest, i don't really care!
