Forum Moderators: open

Message Too Old, No Replies

Mozilla Link Prefetching

Anyone using it in their browser or pages?

         

amoore

5:45 pm on Oct 17, 2002 (gmt 0)

10+ Year Member



I notice that with the release of Mozilla 1.2 Beta [mozilla.org], they are starting to support link prefetching [mozilla.org]. Essentially, this allows browser idle time to be used to download pages that are expected to be used in the near future (like the next page of a multi-page article). You put tags in your HTML or new headers to indicate what content should be prefetched after a page is loaded.

I was wondering if anyone has started using it in their content and has noticed it used by any users, or if anyone has turned it on in their browsers and noticed it doing anything. I understand it's one of those deals that both parties have to start using, so it's unlikely that it's caught on enough to begin to work, but any experiences would be interesting.

dingman

6:19 pm on Oct 17, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Looks interesting. I might start using it, but I'm a bit disappointed that there's no mechanism for telling the browser that it should pre-fetch even though there is a query string in the URL. The site I would find it most useful to use the feature on is generated from a database at each request, but for any given message ought to be safely cacheable.

Do any browsers other than Mozilla support this feature? I have a higher-than-usual proportion of alternative browser users, but even so Moz 1.2b isn't a blip on the radar.

Brett_Tabke

3:42 am on Oct 18, 2002 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



I find the feature mildly disturbing. If it catches on, and Moz does away with the required meta tag and blindly starts to dl links, bandwidth usage could skyrocket.

On the other hand, those that rely on advertising could find the feature most profitable. Cram those links onto the home page and rest assured of a few dls per moz user - money in the pocket.

rjohara

4:09 am on Oct 18, 2002 (gmt 0)

10+ Year Member



Do any browsers other than Mozilla support this feature?

The beta versions of the Mac browser iCab [icab.de] have had a pre-fetch feature for a while. When I was experimenting with iCab I used this feature a little, but it generated so much churning in the background that I left it off most of the time.

Purple Martin

4:48 am on Oct 18, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I'd switch it off. I'm happy to pay my ISP for downloading the pages I choose to view, but I'm not going to pay for downloading pages I haven't even requested. If a future browser won't let me switch it off, I'll change to one that does.

netcommr

6:26 am on Oct 18, 2002 (gmt 0)

10+ Year Member



Does Mozilla look at anything other than below?

HTML 4.01 Specification [w3.org]

For instance, links defined by the LINK element may describe the position of a document within a series of documents. In the following excerpt, links within the document entitled "Chapter 5" point to the previous and next chapters:

<HEAD>
...other head information...
<TITLE>Chapter 5</TITLE>
<LINK rel="prev" href="chapter4.html">
<LINK rel="next" href="chapter6.html">
</HEAD>

The link type of the first link is "prev" and that of the second is "next" (two of several recognized link types). Links specified by LINK are not rendered with the document's contents, although user agents may render them in other ways (e.g., as navigation tools).

keyplyr

9:04 am on Oct 18, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I've been using <link rel="next".... for about 3 years now and has been supported by IE4+ to my knowledge.

Brett_Tabke

12:26 pm on Oct 18, 2002 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



Anyone experimenting with this? (I've not had time yet). Can you specify documents on another site? If so, affiliate programs will have a big boom time! Load the destination page as a prefetched document - the affiliate cookie gets set in doing so, and all that is left is a trip to the bank.

jtoddv

1:13 pm on Oct 18, 2002 (gmt 0)

10+ Year Member



Also with talk about Cable and DSL companies switching to a price per usage model, instead of flat fee, I don't think a lot of people would like this.

rogerd

1:58 pm on Oct 18, 2002 (gmt 0)

WebmasterWorld Administrator 10+ Year Member



If use increases, this will also make web stats meaningless. You won't know if a page was really viewed, or whether it was just preloaded.

gsx

2:29 pm on Oct 18, 2002 (gmt 0)

10+ Year Member



Why all the fuss? NetAccelerator (and other programs) do exactly the same and have been around for years... speeds up some browsing immensly, but sites with lots of links on a page dont seem to have much effect on performance.

tedster

4:54 pm on Oct 18, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



If use increases, this will also make web stats meaningless.

That would be very bad - throwing away one of the most valuable aspects there is in web marketing.

However, it certainly is a potential. Many businesses make only the most minimal use of their server logs - they just haven't got a handle on the value yet. But I've worked with businesses who've fine-tuned or even redirected their offline business plans based on what they've learned online.

I'd hate to see that potential be diminished - especially when broadband obviates the need for most preloading.

Brad

5:00 pm on Oct 18, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I am not sure I would want to run Moz with this. Can this feature be switched off?

Duckula

5:04 pm on Oct 18, 2002 (gmt 0)

10+ Year Member



At least on this case link prefetching doesn't send a referer header; look out the faq at [mozilla.org...]

Do prefetched requests contain a Referer:header?
No, prefetched requests do not contain a HTTP Referer: header. This is because the user has not explicitly requested the prefetched document. This certainly may impact referrer tracking that is commonly used on many sites. For this reason, link prefetching may not be appropriate for all content.

It should be easy to filter prefetched requests.

<added>
user_pref("network.prefetch-next", false); on your user.js switches it off. </added>

<added2> Ouch. It's going to be impossible to determine if a prefetched pages was viewed or not... </added2>

[edited by: Duckula at 5:10 pm (utc) on Oct. 18, 2002]

dingman

5:08 pm on Oct 18, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



We are considering adding UI for this preference; however, the overriding theory is that if link prefetching needs to be disabled then there must be something wrong with the implementation.

Looks like we'll have to get used to filtering out requests with no referrer in our stats. After all, how many users are going to edit their prefs.js? Heck, I consider myself a programmer before a web-anything, and I've never been motivated enough to look at prefs.js.

Brett_Tabke

2:32 am on Oct 19, 2002 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



Well, I think I'm going to take a stand on this thing and add Mozilla to our htaccess ban list network wide.

I certainly wouldn't take such a decision lightly, but Moz is out in left field putting an auto spider in surfers hands. I also agree that it is a user feature - we _need_ more user features in browsers today to handle the hostile web. Unfortunatly, this feature does nothing to make the browser easier to use with so many trashed out pages out there. It just causes server abuse and it's utility to users is marginal.

Any one with a counter opinion on why we should allow mozilla to cause such server abuse?

Duckula

4:30 am on Oct 19, 2002 (gmt 0)

10+ Year Member



Any one with a counter opinion on why we should allow mozilla to cause such server abuse?

I'm not going to excuse such a design faux pas. But Brett, I'll keep using latest Mozilla builds, I'm going to keep coming here, I'm switching off that feature at user.js, and if you keep me out you're going to see a few odd useragent strings at your logs :)

dingman

7:09 am on Oct 19, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



If you feel that strongly, I'll edit my prefs.js, but I'm not giving up Mozilla. If Opera comes out of left field with an announcement that they've open-sourced Opera7, I'll think about it, but there's nothing else half as good as Mozilla and its derivatives for my platform.

Besides, there's no <link>ing in the <head> of this page...

netcommr

8:02 am on Oct 19, 2002 (gmt 0)

10+ Year Member



https:// URLs can be prefetched [mozilla.org]

They actually download secure information and cache it without users authorization or control unless the user has read that page and possibly can figure out what a prefs.js is.

That is wrong!

I hope this doesn't filter over to the next Netscape.

amoore

9:30 pm on Oct 19, 2002 (gmt 0)

10+ Year Member



Bandwidth use by Mozilla users is up 30% the last day.

Perhaps because a new version of mozilla is out. I wouldn't attribute it to the prefetching necessarily. If you ban Mozilla, I'll have to change my useragent string, use Konqueror (or some other linux client), surf webmaster through google's cache, go back upstairs to IE on my mac, or just quit coming around. I can't imagine you wanting your users to have to do any one of those.

Perhaps Mozilla needs to change its useragent string when it's prefetching so you can refuse those queries, or it needs to obery robots.txt or something. That would be great.

I am rather excited about link prefetching, and I'd like to see it work. Perhaps I'll direct the mozilla developers to this thread so that they can see some of the discussion.

bird

10:08 pm on Oct 19, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Any one with a counter opinion on why we should allow mozilla to cause such server abuse?

Are you sure you thought this one through carefully enough?

Mozilla will prefetch a page only when another page contains an explicit prefetch link to it. The user has no way of telling Mozilla "please prefetch all links on this page". It is all in the hands of the webmaster.

If anything, then 30% increase in Mozilla bandwidth use is a sign that more and more web savvy people are switching to decent and up to date client software. If you want to keep those folks away from here, then you know what to do... ;)

[edited by: bird at 10:11 pm (utc) on Oct. 19, 2002]

Brad

10:09 pm on Oct 19, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Banning Moz? I'm having to use Moz on this Mac until Opera comes out of beta.

I'd hate to have to use IE. *ick* :)

bird

10:15 pm on Oct 19, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



https:// URLs can be prefetched

They actually download secure information and cache it without users authorization or control unless the user has read that page and possibly can figure out what a prefs.js is.

This is not "secure information". This is information that was transmitted between the server and the client in a way that is supposed to prevent interception by a third party. If the server maintainer decides that it is ok to allow caching this information, why should that be a problem for the user?

Duckula

9:52 pm on Nov 27, 2002 (gmt 0)

10+ Year Member



The status of prefetching changed. The final Mozilla 1.2 has been released and the link prefetching FAQ was updated:

Do prefetched requests contain a Referer:header?
Yes, prefetched requests include a HTTP Referer: header indicating the document from which the prefetching hint was extracted.

As a server admin, can I distinguish prefetch requests from normal requests?
Yes, we send the following header along with each prefetch request:

X-moz: prefetch