Forum Moderators: open
In this connection, has anyone read 2004 WWW Conference paper "Sic Transit Gloria Telae: Towards an Understanding of the Web’s Decay" by Ziv Bar-Yossef et alii? (URL: [wwwconf.ecs.soton.ac.uk...] It presents a concept of "decay" richer than "are there broken links." For example, although Yahoo gets rid of broken links almost immediately, their directory links to many pages which *themselves* link to broken pages, or reside in "neighborhoods" with high concentrations of broken links. They also have good ideas about detecting soft failures, broken pages where you don't get a 404.
Last question. Has anyone observed how Googlebot deals with dead pages? Does it try again every time it spiders the site, or does it stop trying?
[webmasterworld.com...]