Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Digsby IM Enables Web Crawlers Control of Your PC & Bandwidth


mack - 5:41 pm on Sep 8, 2009 (gmt 0)


We have built bandwidth-monitoring technology into our crawler. 80legs will never use more bandwidth than what a given computer's bandwidth cap is. We keep up-to-date records on current ISP bandwidth plans and caps and only use computers that are using an ISP for who we know the plan. We will never use a computer in a way that risks going over the cap.

How can you be sure of this, how can you tell how close a user is to reaching their limit, if they have been surfing, downoading mp3's ect. I can see what you mean about not being the only reason a user will go over their limit, but you can't say for certainty that you won't push them over.

Mack.


Thread source:: http://www.webmasterworld.com/search_engine_spiders/3986022.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com