homepage Welcome to WebmasterWorld Guest from 54.211.47.170
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / WebmasterWorld / Website Analytics - Tracking and Logging
Forum Library, Charter, Moderators: Receptional & mademetop

Website Analytics - Tracking and Logging Forum

This 87 message thread spans 3 pages: 87 ( [1] 2 3 > >     
Google Enables FireFox Prefetching
GeorgePSmith




msg:904116
 2:41 pm on Mar 31, 2005 (gmt 0)

Today Google announced on their blog (it won't let me post a link, search for it) that they now offer "Enhanced searching with Firefox". This boils down to instructing Firefox to "prefetch" [google.com] search results, meaning that every time I search on Google I will be visiting all sorts of web sites that I do not want to visit. Getting all sorts of cookies that I don't want to get. Does anyone else feel that this is a huge violation of online privacy? Or am I just paranoid? :)

-gps

 

zCat




msg:904117
 2:55 pm on Mar 31, 2005 (gmt 0)

This is the first I've heard of this "prefetch" option and boy is it a little worrying. This should be an option and not enabled by default, neither in Google nor in Mozilla and co.

I don't like it at all.

You can turn this off in Mozilla, see:
[mozilla.org...]

How long before someone finds a way to misuse this?

outrun




msg:904118
 2:57 pm on Mar 31, 2005 (gmt 0)

In Firefox, you can disable prefetching by doing the following:

1. Type "about:config" the address bar.
2. Scroll down to the setting "network.prefetch-next" and set the value to "False".

Don't you love all these extra features you have to opt out of.

zCat




msg:904119
 3:04 pm on Mar 31, 2005 (gmt 0)

Webmasters might like to read this too:
[google.com...]

Prefetching may impact your site because the prefetch request will happen whether or not the user clicks on the result, so it may result in additional traffic to your web server.

OK, not that I've got many pages that appear in the top spot at google...

Also:
To block or ignore prefetch requests (from Google and other web sites), you should configure your web server to return a 404 HTTP response code for requests that contain the "X-moz: prefetch" header.

So will the Mozilla-based browser cache the 404 response and not send a further request when the Google user does click on the top link?

Brett_Tabke




msg:904120
 3:12 pm on Mar 31, 2005 (gmt 0)

Anyone have a handy .htaccess ban?

zCat




msg:904121
 3:14 pm on Mar 31, 2005 (gmt 0)

Is there a way to detect a Google Toolbar enabled FireFox?

Not sure; but the issue here affects all Mozilla-based browsers, as the prefetch thing is a standard "feature". Seems to have been in existence since at least 2002.

Brett_Tabke




msg:904122
 3:15 pm on Mar 31, 2005 (gmt 0)

(oops sorry zcat - I updated mine while you were replying there).

This work?

RewriteCond %{X-moz} ^prefetch [OR]
RewriteRule ^.* - [F]

Craig_F




msg:904123
 3:20 pm on Mar 31, 2005 (gmt 0)

Anyone have a handy .htaccess ban?

I'll second that. I do not like the sounds of this at all. Google is very adamant about us not causing unnecessary loads on their servers, and they think something like this is ok?

carguy84




msg:904124
 3:27 pm on Mar 31, 2005 (gmt 0)

For people who use the google toolbar, does google collect information about 404s? If we setup our servers to return 404s given a certain header, will the sites be penalized at all down the road?

I don't like this one bit...maybe if it was still 1995 when everyone was on dial up...but exactly how much faster do already fast loading pages have to load :(

I wish this could be handled in robots.txt

Chip-

Hanu




msg:904125
 3:44 pm on Mar 31, 2005 (gmt 0)

RewriteCond %{X-moz} ^prefetch [OR]
RewriteRule ^.* - [F]

Brett, are you sure about the [OR]? I think it's redundant.

Brett_Tabke




msg:904126
 4:03 pm on Mar 31, 2005 (gmt 0)

>redundant

a bit, but:

RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR]
RewriteCond %{HTTP_USER_AGENT} ^Hatena [OR]
RewriteCond %{HTTP_USER_AGENT} ^Sleipnir [OR]
RewriteCond %{HTTP_USER_AGENT} ^amzn_assoc [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [OR]
RewriteCond %{HTTP_USER_AGENT} ^autoemailspider [OR]
RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro [OR]
RewriteCond %{HTTP_USER_AGENT} ^URLSpiderPro [OR]
RewriteCond %{HTTP_USER_AGENT} ^Crescent [OR]
RewriteCond %{HTTP_USER_AGENT} ^ContentSmartz [OR]
RewriteCond %{HTTP_USER_AGENT} ^CherryPicker [OR]
RewriteCond %{HTTP_USER_AGENT} ^[Ww]eb[Bb]andit [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebEMailExtrac.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCopier.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^NICErsPRO [OR]
RewriteCond %{HTTP_USER_AGENT} ^Openfind [OR]
RewriteCond %{HTTP_USER_AGENT} ^grub-client [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebWhacker [OR]
RewriteCond %{HTTP_USER_AGENT} ^gigabaz [OR]
RewriteCond %{HTTP_USER_AGENT} ^IEAutoDiscovery [OR]
RewriteCond %{HTTP_USER_AGENT} ^Simple [OR]
RewriteCond %{HTTP_USER_AGENT} ^PingALink [OR]
RewriteCond %{HTTP_USER_AGENT} ^LexiBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^Link*Sleuth [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline*Explorer [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport [OR]
RewriteCond %{HTTP_USER_AGENT} ^Zeus.*Webster [OR]
RewriteCond %{HTTP_USER_AGENT} ^Microsoft.URL [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget [OR]
RewriteCond %{HTTP_USER_AGENT} ^grub [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCapture [OR]
RewriteCond %{HTTP_USER_AGENT} ^Sweeper [OR]
RewriteCond %{HTTP_USER_AGENT} ^Robot [OR]
RewriteCond %{HTTP_USER_AGENT} ^Aide [OR]
RewriteCond %{HTTP_USER_AGENT} ^larbin [OR]
RewriteCond %{HTTP_USER_AGENT} ^Szukacz [OR]
RewriteCond %{HTTP_USER_AGENT} ^httpdown [OR]
RewriteCond %{HTTP_USER_AGENT} ^MSIECrawler [OR]
RewriteCond %{HTTP_USER_AGENT} ^LinkWalker [OR]
RewriteCond %{HTTP_USER_AGENT} ^sitecheck.internetseer.com [OR]
RewriteCond %{HTTP_USER_AGENT} ^ia_archiver [OR]
RewriteCond %{HTTP_USER_AGENT} ^Seeker [OR]
RewriteCond %{HTTP_USER_AGENT} ^ASPSeek [OR]
RewriteCond %{HTTP_USER_AGENT} ^DIIbot [OR]
RewriteCond %{HTTP_USER_AGENT} ^Indy*Library [OR]
RewriteCond %{HTTP_USER_AGENT} ^psbot [OR]
RewriteCond %{HTTP_USER_AGENT} ^foobot [OR]
RewriteCond %{HTTP_USER_AGENT} .*almaden.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^MSProxy [OR]
RewriteCond %{HTTP_USER_AGENT} ^SlySearch [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper/2.09 [OR]
RewriteCond %{HTTP_USER_AGENT} ^EasyDL/2.99 [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZip [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Missigua [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebAccounting [OR]
RewriteCond %{HTTP_USER_AGENT} ^HTTrack [OR]
RewriteCond %{HTTP_USER_AGENT} ^Dumbot [OR]
RewriteCond %{HTTP_USER_AGENT} ^NaverRobot [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper [OR]
RewriteCond %{HTTP_USER_AGENT} ^Xenu [OR]
RewriteCond %{HTTP_USER_AGENT} ^Seekbot [OR]
RewriteCond %{HTTP_USER_AGENT} ^InternetSeer [OR]
RewriteCond %{HTTP_USER_AGENT} ^tScholarsBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^SearchIt.Bot [OR]
RewriteCond %{HTTP_USER_AGENT} ^abot [OR]
RewriteCond %{HTTP_USER_AGENT} ^User-Agent [OR]
RewriteCond %{HTTP_USER_AGENT} ^mogren [OR]
RewriteCond %{X-moz} ^prefetch [OR]
RewriteCond %{HTTP_X_FORWARDED_HOST} ^world.altavista.com [OR]
RewriteCond %{HTTP_X_FORWARDED_SERVER} ^altavista.com [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailCollector
RewriteRule ^.* - [F]

works too ;-)

GoogleGuy




msg:904127
 4:14 pm on Mar 31, 2005 (gmt 0)

Note that even if your show up at #1, this tag still won't be shown most of the time. It's only shown if we're pretty sure that the user will click on the top result. So if webmasterworld.com is #1 for the search "webmaster world" then we might add the prefetch tag. But we wouldn't if WebmasterWorld were #1 for another search. It can actually make your site look like it loads faster; I'd give it a week before you assumed that it created new load..

mattglet




msg:904128
 4:26 pm on Mar 31, 2005 (gmt 0)

...if we're pretty sure...

That's amusing.

Brett_Tabke




msg:904129
 4:29 pm on Mar 31, 2005 (gmt 0)

> if we're pretty sure...

So you guys are using a background algo to analyze click behavior and only show the tag if there is a high propensity to click on the top result? Interesting.

In other news, how much you wanna bet FireFox and mozilla numbers are up next month?

walkman




msg:904130
 4:33 pm on Mar 31, 2005 (gmt 0)

"So you guys are using a background algo to analyze click behavior and only show the tag if there is a high propensity to click on the top result? Interesting. "

More like if the search term matches the domain name, or variations of it: webmaster world,webmasterworld, web master world,webmasterworld.com

freshfish




msg:904131
 4:42 pm on Mar 31, 2005 (gmt 0)

HEllo,
Sorry if this sounds like a noob question; but what does this htaccess do that Brett posted?
...
RewriteCond %{HTTP_USER_AGENT} ^User-Agent [OR]
RewriteCond %{HTTP_USER_AGENT} ^mogren [OR]
RewriteCond %{X-moz} ^prefetch [OR]
...
Is it a way to block these agents?

Thanks

zCat




msg:904132
 4:43 pm on Mar 31, 2005 (gmt 0)


It's only shown if we're pretty sure that the user will click on the top result.

GoogleGuy: regardless of whether you're pretty sure I'll click on the top result, how do you know I want the page to be prefetched? I'm on broadband so the few ms that might be saved are irrelevant. And the times I'm likely to be on dialup are also times when I will probably need the bandwith for stuff I _definitely_ want to load.

BTW: I think prefetch can be a useful idea, but should be an optional extra for those who want / need it. A large part of the problem is not so much Google using it, but Mozilla for enabling it by default.

Brett_Tabke




msg:904133
 4:44 pm on Mar 31, 2005 (gmt 0)

a bit of a side topic there freshfish, but start reading here: [webmasterworld.com...]

GeorgePSmith




msg:904134
 4:46 pm on Mar 31, 2005 (gmt 0)

Thanks for the clarification GoogleGuy, but this still feels like a rather large violation of my online privacy. Don't get me wrong, the concept of prefetch sounds like a good thing, but the side effects of this are just plain scary. I don't want my traffic going to a site unless I click the link. Plain and simple. This prefetch change can place unwanted cookies on my computer for sites that I have never visited. I fear that this can somehow be exploited. I may sound paranoid, but that is why I switched to Firefox in the first place! :)

All kidding aside, I've been conducting a few tests and I honestly don't see a difference in load times. It appears as though Firefox is only prefetching the HTML, not the images, scripts, etc on the site. Those tend to be the slower items, not the HTML of the page itself. To me, this appears to do too little to speed up the load times and can quite possibly be exploited. I'd like to see this "feature" removed from Google ASAP.

diddlydazz




msg:904135
 4:51 pm on Mar 31, 2005 (gmt 0)


Getting all sorts of cookies that I don't want to get. Does anyone else feel that this is a huge violation of online privacy?

My thoughts too.

Unless Google can control which sites it applies to, then the cookie issue is only part of it.

All this will do IMO as webmasters is mess up our stats.

Surely Google's time would be better spent concentrating on other issues at the moment.

The prefetch feature was designed to prefetch pages from links on a page that the user requested, not what some SE *thinks* they want.

ronburk




msg:904136
 5:02 pm on Mar 31, 2005 (gmt 0)

I do not like the sounds of this at all.

Oh, it's simply dreadful! The thought that a Google searcher will see my page pop up faster has left me nearly inconsolable!

I'm on broadband so the few ms that might be saved are irrelevant.

On this point, I suggest Googling for "I don't know anything about typical latency delays on the Internet, or even about the cost of DNS fetches for first-time visitors."

And, in the complaining-to-the-gas-station-attendant-about-the-price-of-gas department:

I'd like to see this "feature" removed from Google ASAP.

Repeat after me: the "feature" is in the browser, not in Google. Getting that straight means you can just change one config setting instead of spending the rest of your life complaining to an infinite number of websites who elect to insert this particular browser "hint" in their HTML.

Meanwhile, requests for favicon.ico continue to beat the Internet to death with no particular value added. Who do I see about that?

:-)

GoogleGuy




msg:904137
 5:02 pm on Mar 31, 2005 (gmt 0)

I knew folks here would be skeptical--I even asked them to change the FAQ at [google.com...]
to emphasize that this only happens when we have pretty high confidence--not every #1 result gets this. I would say to give it a chance first; give it a few days and then check your server logs to look at the level of prefetches, because I think it will be quite low. If you give it a chance and don't like it, the url above tells how to distinguish a prefetch request from a normal request and how to block it if you want. All I'm asking is for people to give it a try for a few days before they decide whether to block it or not.. :)

Brett_Tabke




msg:904138
 5:06 pm on Mar 31, 2005 (gmt 0)

You know - GG is dead right - this isn't a Google issue. The issue is Mozilla including support for what is esentially a site scrapper tag. It is time to take this up with Mozilla.

[edited by: Brett_Tabke at 6:25 pm (utc) on Mar. 31, 2005]

moltar




msg:904139
 5:06 pm on Mar 31, 2005 (gmt 0)

This makes sense. Google uses tracking URLs/JavaScript for quite a while now. I am sure they compiled a lot of information regarding what people do and don't click. I am sure it will be for our users' benefit. I'd wait and see the actual results.

P.S. I am think to prefetch AdSense URLs too for the benefit of my users. j/k

Brett_Tabke




msg:904140
 5:08 pm on Mar 31, 2005 (gmt 0)

> P.S. I am think to prefetch AdSense URLs too
> for the benefit of my users.

Oh the irony! nice one. :-)

> pad logs

With Google Referrals and Mozilla agent numbers. Hence, Mozilla will falsly jump in the agent name market and Google will jump in the referral dept.

very smoooth...

zCat




msg:904141
 5:23 pm on Mar 31, 2005 (gmt 0)


I'm on broadband so the few ms that might be saved are irrelevant.

On this point, I suggest Googling for "I don't know anything about typical latency delays on the Internet, or even about the cost of DNS fetches for first-time visitors."

'Your search - "I don't know anything about typical latency delays on the Internet, or even about the cost of DNS fetches for first-time visitors." - did not match any documents.'

;-). I don't know about typical latency delays, but for me everything's just fine as it is. "Prefetching" DNS lookups might be a nice idea though.


And, in the complaining-to-the-gas-station-attendant-about-the-price-of-gas department:

I'd like to see this "feature" removed from Google ASAP.

Repeat after me: the "feature" is in the browser, not in Google. Getting that straight means you can just change one config setting instead of spending the rest of your life complaining to an infinite number of websites who elect to insert this particular browser "hint" in their HTML.

This is the crux of the problem: an obscure (I'd never heard of it until now) browser function (in Mozilla) with all kinds of potential for causing Things To Happen That Shouldn't, which defaults to "on", which means for the rest of my life I'll have to remember to switch this off whenever I'm using a "fresh" browser.

Thanks to Google to bringing it to my attention ;-).

karmov




msg:904142
 5:33 pm on Mar 31, 2005 (gmt 0)

Silly question I suppose, but I just want to confirm that this identifier exists only in the http header. Does this mean that there's no way to sort out the difference between a prefetch and a visit after it's made its way into my logs?

I've tried running a test but my site's not cool enough to get a prefetch tag as of yet for its main identifying phrase :)

diddlydazz




msg:904143
 5:42 pm on Mar 31, 2005 (gmt 0)


the "feature" is in the browser, not in Google

the feature is for users to prefetch pages from links on a page that they have requested and probably trust.

Unless Google can guarantee that there will be no dodgy tactics used by the site at number one for any particular search then i can see this being exploited.

(the "oh, but they requested the SERP" argument doesn't stand up)

It is not designed so a search engine can guess which link the user wants and prefetch the code!

If that was the case then the "im feeling lucky" would be the default search

or am i just missing something really useful? :o)

(I admire google for taking the initiative, i just can't see the benefit over the potential problems)

steve40




msg:904144
 6:20 pm on Mar 31, 2005 (gmt 0)

I can see a major use of this not for the good of the user

The first result on G could just happen to be adwords premium spot and G or another SE decides to prefetch this , this seems like could be used like popups were in the old days

The theory seems good but the possible abbuses make it a no go as far as i'm concerened

also as others have pointed out graphics etc. are generally the reason a page loads slow and those not prefetched sorry don't like or trust reasons for implementation

steve

moltar




msg:904145
 6:23 pm on Mar 31, 2005 (gmt 0)

It's actually a useful feature if used right. Think about:
  • Online versions of presentations
  • Picture galleries
  • Articles with multiple pages
  • Maybe even blogs with new entries
  • Any other "linear" browsing experience

I am sure there are way more useful applications. Some gallery scripts were doing this with JavaScript even before the feature was released by Mozilla.

Remember - user experience is key. If user thinks your site is way faster then your competitor's site - they will come back to you. Read "The Need for Speed [useit.com]" by Jakob Nielsen.

Research on a wide variety of hypertext systems has shown that users need response times of less than one second when moving from one page to another if they are to navigate freely through an information space. Traditional human factors research into response times also shows the need for response times faster than a second. For example, studies done at IBM in the 1970s and 1980s found that mainframe users were more productive when the time between hitting a function key and getting the requested screen was less than a second.

[edit]added quote[/edit]

[edited by: moltar at 6:31 pm (utc) on Mar. 31, 2005]

This 87 message thread spans 3 pages: 87 ( [1] 2 3 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / WebmasterWorld / Website Analytics - Tracking and Logging
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved