Google Podcast: "Crawling smarter, not harder"

[engine]

The Google team talk about, "misconceptions around crawl frequency and site quality, what's challenging about crawling the web nowadays, and how search engines could crawl more efficiently."