Page is a not externally linkable
- Search Engines
-- Ask - Teoma
---- Ask eating bandwidth

Dijkgraaf - 9:38 pm on Jul 7, 2005 (gmt 0)


You could slow it down by putting a high Crawl-Delay for that bot. You could also set expiries for pages well into the future so it might crawl those pages left often.

Thread source:: http://www.webmasterworld.com/ask_jeeves_teoma/504.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com