| 3:21 pm on Nov 15, 2006 (gmt 0)|
That is the generic message that explains why the faster option is available to you. I see that it is somewhat confusing to still be there *after* you have chosen faster. I'll work on making that less confusing.
| 11:02 pm on Nov 15, 2006 (gmt 0)|
Yeah...... it *does* seem confusing. I would just change the wording. The "faster" option had been set for a while so I was thinking the wording meant that sinc emy page load times for lots of random fetches were high (long story) that Google was throttling me even though I had sent it to Faster.
| 11:09 pm on Nov 15, 2006 (gmt 0)|
um, i don;t think this is the generic message. I think you;ve set it to faster, google reckons your server can;t handle it, so its slowed down.
If you now choose to drop back down to normal, I doubt that the faster option will be available to you
| 12:16 am on Nov 16, 2006 (gmt 0)|
Ah, good timing!
I was just getting the results from the "experiment" we've started in this thread:
Right after crawl rate showed up in the WM tools...
back then (Oct 17th 06) I've had these stats:
Pages: Max/6310 - Avg/1796
Kb: Max/334995 - Avg/79489 (80Mb per day!)
Mil sec: Max/3063 - Avg/1749
After setting it to "faster" - today (Nov 15th 06), I have these:
Pages: Max/6310 - Avg/2524
Kb: Max/334995 - Avg/101801 (100Mb per day!)
Mil sec: Max/3421 - Avg/1794
So in general per day I've got Google to spyder 728 MORE pages per day! (... or it seems that way)
Looking at the graph I notice that google spydering slowed down considerably from mid-September to mid-October, which could count towards lower average number on my original post (the tracking starts from mid-August) - so the spydering from mid-Aug to mid-Sept is about the same as from mid-Oct to mid-Nov...
Conclusion: I still don't know if setting it to faster had actually made it faster... it certainly seems to be the case when comparing the last 2 months... but it's only as fast as it was 4 months ago...
| 12:32 am on Nov 16, 2006 (gmt 0)|
My minimum number of pages crawled has actually dropped slightly since I increased the crawl speed to "Faster".
| 12:38 am on Nov 16, 2006 (gmt 0)|
In my opinion the minimum number doesn't mean much... since we had a couple of outages - and I'm sure Google couldn't possibly maintain the sypder crawling non-stop all the time either... so it's kind of moot point...