| 6:15 am on Oct 23, 2003 (gmt 0)|
It's all normal, and it doesn't matter if it's cpanel, it depends on whether your host makes logs available to you, or what kind of stats program you've got. That's how you'd see.
| 9:17 pm on Oct 23, 2003 (gmt 0)|
nuevojefe if you can get your raw logs, google's crawler asks for robots.txt regularly (it you dont have one of these this log entry may be in your error log)
Plus it usually always leaves a referrer saying something like
or similar, once you have some of these its easy to isolate what it has requested.
| 3:16 pm on Oct 24, 2003 (gmt 0)|
Thanks for the pointers, I think my raw log was turned off, I changed it yesterday to auto-archive so I'll see if it's not empty anymore in a minute.
| 8:13 pm on Oct 24, 2003 (gmt 0)|
The most helpful I found, was using a custom 404 page that sends an e-mail.
Thus I see any stupid mistakes I make and as I don't have a robots.txt all the crawlers looking for it
See if your cpanel allows you to define a custom error page and if yes plug a nice error message in that also sends an e-mail (look on g for custom error pages).
| 4:23 pm on Oct 27, 2003 (gmt 0)|
I put <!--#echo var="DATE_GMT" --> in the footer of my pages. When Google caches the page it will show the exact date and time that the page was crawled.
| 8:03 pm on Oct 27, 2003 (gmt 0)|
Great idea about the date...
Anyone know how to code this same thing in asp?
<!--#echo var="DATE_GMT" -->
| 8:16 pm on Oct 27, 2003 (gmt 0)|
Nevermind the post I made above I used:
<% Response.Write FormatDateTime(Time, vbLongTime) %>
<% Response.Write FormatDateTime(Date, vbLongDate) %>
| 4:04 pm on Oct 29, 2003 (gmt 0)|
For those who are interested: In php files I use the following in the footer area to catch the spider date and time in the cached copy of the page:
<?php echo date("l dS of F Y h:i:s A T");?>
On another topic,using the line below in the second line of a php file sets the file header information to show "recently modified" and can help with fresh listing:
header('Last-Modified: '.gmdate('D, d M Y H:i:s \G\M\T', time()));