Welcome to WebmasterWorld Guest from 188.8.131.52
Has this been disussed before here?
If not, I have two questions:
1) googlebot alone seems to be responsible for some 5-10% of my overall traffic volume. Is this normal?
2) The average crawl-speed lies around some 1000 milliseconds per page. I think this is quite OK? But there were some peaks with up to 4000 milliseconds. What are the recommended benchmarks concerning eg. adwords-quality-score?
For question #2, I'm not much of a server benchmark guy - sorry. Some server benchmarking I've seen indicates that over 1,500 ms generates a lot of user complaint. As a general rule, I've found that Adwords QS imrpoves tremendously as load speed goes up, and I'm beginning to suspect that the same is a factor in organic ranking. And page-load speed involves a lot more than server response time.
Maybe someone else has some idea about server benchmarking?
"Make sure your web server supports the If-Modified-Since HTTP header. This feature allows your web server to tell Google whether your content has changed since we last crawled your site. Supporting this feature saves you bandwidth and overhead."
Maybe its time to implement that.
I assume it has nothing to do with the date-last-modified-entry on harddrive? I use a lot of scripting and even my .html-pages are parsed as php-files. So it wouldn't make sense to deliver a 304 according to datelastmodified if some of the scripts involved have undergone changes meanwhile. And I think I'm not the only one, who does so.
I have a managed server on a well-established larger hosting-company and I assume they have implemented their own cache-system in order to reduce processor capacities, thereby delivering a 304 by default whereever appropriate. I might try to find out about this exactly via mail, but maybe you know, how this is generally handled?
I took a closer look at the figures:
The overall traffic in megabytes amounts to 2-2,5% for the figures given in GWT. As for the number of pageviews googlebot is crawling approximetely one third of my pages every day, which amounts to 5% of all pageviews. I think the difference as compared to traffic volume results from the fact that googlebot does not request images?
The average load-speed lies around 1200 milliseconds and the GWT specifies this as complete pageload-time, not server response-time. I think I can live with that.
I still use a table-based design, but with a php-flush() command after the header-table, which thus loads very quickly. Due to a varying amount of db-requests (up to 40 products per page), some pages (tables!) of course take a bit longer to load but we never had any visitor-complaints concerning load-speed whatsoever.
> I've found that Adwords QS imrpoves tremendously as load speed goes up, and I'm beginning to suspect that the same is a factor in organic ranking.
This is what I also suspect and it would be very interesting to get some details on this.
> regional average plus 3 seconds.
I think I'm far better than that.
I hope the figures I gave might help others to narrow down things a bit. So far two questions remain for me
1) what the hell is "regional average"?
2) Since when does GWT show these graphics? I think the last time I visited that page before was in autumn last year.