Forum Moderators: open
teeceo.
'Freshbot' seems to be on a 'PR' basis.
That being said, the links to other websites will help the PR somewhat, but not necessarily enough to achieve freshness. You could try, but it may not be enough.
teeceo.
I'm not too sure about that. Our main site is PR<1 (reads 0). Freshbot visits on a regular basis and many of our pages have fresh tags -- if you can find them.
From the very little I know and have observed, freshbot does not do much deep crawling. It hits the page, checks to see if it has changed, then moves on. It does not look like it follows links -- nor does it seen to cache -- any of its findings. If it finds a fresh page it indexes it, puts a fresh date on it, that's it.
I think that to get freshbot to do anything your page it already has to be in the index and it must have *changed*. (And there are always the exceptions of new pages).
There was talk some years ago when AV first started "freshness dates." How did you force a spider to reindex your page? Was it by the header date or somethingelse? At that time some more informed people suggested that changing the overall page size would force the spider to re-index, something along the lines of 75 bytes or so.
It sill works.
As for the linking part -- unless you're abusing it it's fine. But to get freshbot to reindex those pages you have to do something to them. If they don't change freshbot will ignore them. Hence its name.
Jim
It does not look like it follows links -- nor does it seen to cache -- any of its findings.
I have a page that I updated last night (just prior to freshbot stopping by) and added links to 10 new pages I just created. Freshbot followed these links and the pages are in the index today with fresh dates, cached and with similar pages (30) listed for them.
I'm an old intelligence wimp (It's *probable* that the bad guys are invading the good guys -- when we're watching them do it!) I always try to qualify, there are just too many dang exceptions.
And -- the pages are NEW! They're FRESH. That's what freshbot wants.
Kind of like: Same old stuff, same old stuff, NEW, same old stuff, same old stuff, CHANGED. Bingo, you've got a new page and a changed page in freshbot.
So let's say this: Freshbot is not a deep crawl. It checks pages that are already in its index. If it finds a changed page it re-indexes it. If it find a new link it follows it and indexes that page.
Makes sense from a resources point of view.
'Freshbot' seems to be on a 'PR' basis.
I have a site that currently shows PR grey bar (still trying to work out if it has been penalised or not, but if it has, no idea why) and Freshbot has visited virtually every day this month.
I look at it this way. Even if I do not fully understand its workings, this is how I would do it if I were Google.
I think Freshbot visits are split in these two categories:
Most important pages frequently and most topical recently.
What are the most important pages that need frequent or recent spidering?
1. Pages that contain content that has recently become very important topically-wise, according to the searchers and the web community at large.
How to determine what those pages are? Best bet would be:
1.1 Pages that got fresh links (recent links) from pages that are considered authorative (high Pagerank). To do that, you need to spider those authorative pages frequently ;)
1.2. Pages that contain content and turn up top 100 in Serps, for search queries, that recently are searched for a lot compared to the average recent past.
Take the generic search data Google has and e.g. shows in a limited way in Zeitgeist.
Google can detect which keyphrases are suddenly occuring more frequently. [wired.com]
2. Does this mean that on-page content should be changed frequently with the above set-up?
No, I would say not initially, but maybe for the authorative Fresh pages to remain Fresh, they should have new or altered links on them once in while?
teeceo.
