Needless page loading

Browsers that load pages even if the user never visits that page.

[UserFriendly]

Is anybody else miffed that some browsers load web pages in advance, whether or not the user ever intends to visit the page at the end of the link?

For example, if you have a series of pages that run in order, it's nice to use <link rel="next" ... /> markup in the head element to declare the structure of your site. It also allows browsers like Opera to show a navigation menu with working "next" and "prev" buttons.

But I'm seeing in my access logs that some requests are loading the page pointed to by the "next" link, even though the visitor never visits the next page.

It all seems like a waste of bandwidth to me. Are browsers starting to turn this behaviour on by default?

[stapel]

I think the process is called "acceleration" and/or "prefetch", and is intended to make for faster surfing. By pre-loading the pages into the cache while the browser is otherwise unoccupied, the page will appear to load more quickly if/when the user actually follows the link.

Yes, this behavior can lead to problems....

Eliz.

[UserFriendly]

When did prefetch become a feature of Firefox, though? Has someone specifically installed an extension to add this feature, or is it now default behaviour?

[stapel]

I think it's a Google plug-in for Firefox. I could be wrong, though....

Eliz.

[Pfui]

The Fasterfox plug-in is the leading pre-fetcher but it will heed robots.txt -- that's why you'll see Firefox browsers requesting the file. Here's [webmasterworld.com] more info, including how to block it.

And for additional info about pre-fetching generally and ways to stop it, here [google.com] you go.

Personally, I think pre-fetching is both a bandwidth drain and a royal pain, particularly with broadband saturation in the U.S. approaching 50% (as I recall reading... somewhere). I simply don't see any reason why people need to pre-load pages they may never read when they can see them near-instantaneously anyway. (grump,grump)

[UserFriendly]

I don't mind people on slow connections using prefetching to make their web experience less miserable. But just enabling this behaviour by default is outrageous. I've lost 3.9MB bandwidth on one page alone today. (I can tell because the page has been requested 189 times, and the images for the page have only been requested 35 times.)

[Pfui]

UserFriendly, I'm not sure what you mean by 'enabling pre-fetching by default.' The Fasterfox plugin, for example, is not included with Firefox, etc. Someone has to want to install it, go find it, get it, install it, configure it, yadda-yadda.

There may be some browsers shipping with a tabbed browsing-related 'load links in the background' pref enabled but I'm not seeing the kind of server-revving hit rates which would result from something so obvious -- and potentially costly -- being a widespread default.

That said, just yesterday I saw a Firefox browser --

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.12) Gecko/20050922 Fedora/1.0.7-1.1.fc4 Firefox/1.0.7

-- request robots.txt (where I have a blanket Disallow), breeze through six .html files in two seconds flat (including 1/3 of a frameset) like it was invited, and then poof. It was gone.

The behavior may have been because of some other plug-in, or some new somethingorother (hope not!), or a spoofed UA. Either way, I was not happy to see the results. And I was surprised to see that the bot-runner's IP belonged not to the usual country suspects but to...

VeriSign Infrastructure & Operations.

Go figure.

.
P.S.
I hope you were able to block your bandwidth hog!