Forum Moderators: open
But now it seems like they improved the freshbot and SSI includes don't count any more for fresh dates.
For example, this single website and many others have been spidered yesterday but didn't get a freshtag today. Seems strange to me. I see this strange freshbot behaviour since monday.
I get pages spidered all the time but they are in the index with a June refresh date.
64.68.82.182 - - [17/Jul/2003:04:19:50 -0400] "GET /web-translation/ HTTP/1.0" 200 2823 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"