Fresh Content

How "speak" Freshbot with my Web server?

[joker197cinque]

Hallo,

I heard that freshbot likes fresh content.

How FBot know this?

What does it ask to my server?

And how my server answer in positive way or in negative way?

[deanril]

Welcome to Webmaster World!

Basicly Fresh bot will find this all its own, or the deep crawl will, the bots look for a "robots.txt" file, on your server, if there is one it will obey the commands (most of the time) that are in there.

Basicly the bot will come buy at will, it will look for the robots file, and thats about all it will talk to your server about, then it will basicly crawl your pages and pic up anything new you have.

I have noticed, Ive been changing my site daily, and fresh bot comes by daily, but deep crawl does not like me yet :(

[BGumble]

The quickest way for freshbot to determine if you have fresh content is by the filesize. If it's more in depth then it would compare the current site to the Google cache and do a % analysis to determine how much change has occured.

Just changing a timestamp or incrementing a counter will likely not be enough to be verified as fresh. See the other Freshbot thread on the list for info on declaring proper validation headers.