Welcome to WebmasterWorld Guest from 126.96.36.199
Forum Moderators: open
I really feel for a former client - 75k pages on two domains (tech manuals). Between the major search engines, he is averaging 200k views a week from the engines. Throw in rogue spiders and it jumps to 500k a week (100gig a month in bandwidth just for spiders).
The litmus test:
If a search engine does more spider page views than it sends out referrals.
Fast will deliver results "round trip" in 600 ms
The question then becomes "how long does the spider take?"
The question then becomes how relivant is your site to the query AND "how long does the spider take?"