I'm getting alot of 304s during this deep crawl and I'm a little bit worried because the pages getting 304s have been updated since the last deep crawl. I don't log the entire http request header so I can't examine the If-Modified-Since dates that deepbot is requesting.
Do deepbot and freshbot share the latest page data or is my server wrongfully returning a 304? I do know for a fact that freshbot has crawled the updated pages since the last deep crawl.
