Will google freshbot follow a link?

google freshbot

[motiv8x]

I'm trying to determine if Google's freshbot will follow a link from one of my sites (which gets indexed by FreshBot daily) to a new site when I put a link there.

I am noticing that the freshbot is appearing in the 2nd site's log files, but I can't determine if it's coming from the first site or not.

There is no "referer" in my log file for the Google fresh bot entry (and all it did was take 1 or 2 pages from the 2nd site).

Does Google freshbot grab the url of site B and then deep crawl it later? or does it get indexed right away?

Any insight is appreciated on how to crosslink sites between sites that are being spidered daily by Google's freshbot!

Thanks!

Motiv8x

[edited by: heini at 12:16 am (utc) on Mar. 7, 2003]
[edit reason] no urls please thanks [/edit]

[Philosopher]

Yes the freshbot will follow links, but there is no reliable way to tell which links it will follow. In other words, the decision is based on freshbots algorithm so it can be hit or miss.

I have used this to get a new site indexed by freshbot, but it doesn't always work.

[vitaplease]

motiv8x,

Think of it this way, IMO the only way for Google to ever find any real Fresh content (that is new pages) is by following links, using toolbar info, or indexing the newly submitted urls.

So they crawl the most important pages very frequently, not necessarily for new content on those pages, but for new links on those pages.

PS. you might want to remove that link, just to keep promotional things within reason (check the TOS).

[netguy]

It's always a good try though... when it works, it's certainly exciting to see those new sites 'pop-up' in Google after only a day or two!