how to get freshbot to crawl deeper?

stays on index..

[matthew]

The google freshbot is and has been on the homepage of our site for quite some time. The only problem is we really need him to go deeper to get at some individual product pages (which are all only one dir deep) Any way to encourage him to go a little deeper? Also, would he be able to improve placement for recently added (i.e. since last big crawl) products?

[jimh009]

Hi Matthew,

One very good way is to get an external link from another site to the pages you want "fresh botted", so to speak. The freshbot hits my site either via the home page or comes in the back door at these deep links (many of which are 5 directory levels down). I suspect the more deep links you have to your site, the more the freshbot will come around and take in stuff. An interesting thing I've noticed is that when the freshbot comes in directly to my home page, it rarely takes in anything else. But when it comes in to other deep pages due to external links, it seems to take in more stuff for whatever reason (as well as grabbing the home page, again).

Just my experiences.

Jim

[troels nybo nielsen]

That is very interesting, Jim. And knowing Google's emphasis on extermal links it indeed sounds logical, that Google would be more willing to let freshbot follow a deep link coming in from outside than your own internal links down in the levels.

Troels

[vitaplease]

I would say it would also help if you can get a deep link to the internal pages from Fresh pages.

Also, add some new internal links to those pages you want to get Fresh-botted.

[Namaste]

what is the approximate amount of time that Google will pick-up up on a link and then act on it?

About 2 months back many external links are pointing to my deep pages and Google has counted them (as can be observed from the Link: command), but I see no change in the actions of fresh bot. It still only goes to my homepage and the the 1st level of links from the homepage