Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Status codes 200 and 304

does googlebot treat them as identical?

         

selomelo

9:08 am on Sep 16, 2006 (gmt 0)

10+ Year Member



I have a dynamic subdirectory on my site. It is a database-driven dictionary site with mod rewrite (output as static html). The subdirectory is updated almost daily adding new entries, modifying/correcting/elaborating existing entries, etc.

However, google seems ignoring all the changes, although it crawls the subdirectory and all the pages almost every day.

I checked my logs, and saw that the pages return a 200 code. But Google does not seem to fetch the file. Here is the typical log entry:

Http Code: 200 Date: Sep 15 19:35:55 Http Version: HTTP/1.1 Size in Bytes: 5
Referer: - Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

What strikes me is the "Size in Bytes: 5" part, since it tells that the file is just checked, and not fetched. Am I wrong?

My log shows thousands of such 200 and size 5 bytes crawls by google.

What does this mean? Does google just checking the file and leaving it alone? Is this 200 in fact a 304 (perhaps generated by php)?

How can I handle this problem? I.e., tell Googlebot that the pages are in fact modified?

Thanks in advance

g1smd

7:19 pm on Sep 16, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I would be tempted to run the pages through [validator.w3.org...] and see that there is not some fatal error on the page which causes spiders to abort.

Next, I would run the site through Xenu LinkSleuth and closely look at the error report that it generates.