Forum Moderators: Robert Charlton & goodroi
on 3/2 and 3/3 they picked up alot of content; did not care if it was old.
on 3/7 and 3/8 they have been sending "if modified since" requests and have gone deeper into my site than normal.
on 3/4 - 3/6 baseline hits.
I've notice the 304 requests because my old style (semi old i'm only 20% converted) used SSI withoug a last modified date. I've been changing content so that I create that header; MSNbot does not use it, but slurp and googlebot seem to like using it. Googlebot seems to hit the 304 rapidly untill it finds a modified page then waits 60 sec till the next hit.
I've been playing with 304 responces because I am working on a dynamic rss feed. And, i thought it would be fun to see how search engines react.