I was just thinking how expensive it must be for Inktomi to spider their paying members' sites every 48 hrs.... when let's face it, most of us do not update our site every 48 hrs.
I bet they would cut their spidering costs down by over 90% and make webmasters much happier if there was a Big Red Index Me! Button So for example, I would press the button and all URL's in my account are refreshed within 6 hrs or something.
I'm happy because I don't get upset if the spider is late after 48 hrs, and I get my content in faster, and they save a lot of money, WIN-WIN!
Msg#: 2004 posted 12:38 pm on Mar 15, 2003 (gmt 0)
Let's say you had just 20 pages in there. In just 10 days, that is 100 variables you could test. You could have some of the pfi systems (all but link pop and the pfi cheat factor) within a couple of weeks of hard testing.