Welcome to WebmasterWorld Guest from 188.8.131.52
Forum Moderators: goodroi
This indicates that nothing is disallowed and the spider can follow all links
To allow a single robot complete access and exclude all others
This would prevent your entire web site from being indexed
Is it normal for Googlebot to crawl just the home page for a week or two (my site was launched on January 15th 2004) and than do deep crawl later. Or this is something unusual?
For the last two weeks Googlebot visits my home page one or twice a day and it crawls only robots.txt and home page.
I have noticed this on the latest sites which I have put online that it has taken a few weeks to do a deep crawl
Just got to hang on in there till it happens