homepage Welcome to WebmasterWorld Guest from 54.198.148.191
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Accredited PayPal World Seller

Visit PubCon.com
Home / Forums Index / WebmasterWorld / Website Analytics - Tracking and Logging
Forum Library, Charter, Moderators: Receptional & mademetop

Website Analytics - Tracking and Logging Forum

This 87 message thread spans 3 pages: < < 87 ( 1 [2] 3 > >     
Google Enables FireFox Prefetching
GeorgePSmith




msg:904116
 2:41 pm on Mar 31, 2005 (gmt 0)

Today Google announced on their blog (it won't let me post a link, search for it) that they now offer "Enhanced searching with Firefox". This boils down to instructing Firefox to "prefetch" [google.com] search results, meaning that every time I search on Google I will be visiting all sorts of web sites that I do not want to visit. Getting all sorts of cookies that I don't want to get. Does anyone else feel that this is a huge violation of online privacy? Or am I just paranoid? :)

-gps

 

steve40




msg:904146
 6:30 pm on Mar 31, 2005 (gmt 0)

I am sorry but the possible abuse could kill firefox
new toolbar for firefox installed by dodgy fellow that changes prefetch from G to anything they wanted

this will be a downhill path and could well hurt the most important alternative browser available which would be truly a shame

steve

insight




msg:904147
 6:33 pm on Mar 31, 2005 (gmt 0)

From a webmasters perspective...

Cons:
-fouls up log/referrer data if pretech requests can only be identified by the http header
-unecessary page requests may hinder site performance for real users

Pros:
+Looking for prefetch tags in SERPs may give some clue into what results are most popular
+Possible to gain insight into what searchers are searching for even if they don't click your link. With advanced tools, could even figure out when your result is getting skipped over.

My opinion:
You can't really stop a user-agent from doing whatever it wants to do, but I would've left this feature off by default in Mozilla (the same way that simultaneous http connections aren't cranked up by default out of courtesy to web servers). Would also be more webmaster friendly if user-agent was modified/different for prefetch requests.

GeorgePSmith




msg:904148
 7:07 pm on Mar 31, 2005 (gmt 0)

give it a few days and then check your server logs to look at the level of prefetches

GoogleGuy: I wish I could. However, this information is not included in the server log. It is included in an obscure HTTP header that isn't logged. While some web servers (like Apache) can log this, others (like IIS) can't.

This isn't a Google issue. The issue is Mozillia including support for what is esentially a site scrapper tag. It is time to take this up with Mozilla.

Brett: I agree that Mozilla should take some heat here. Hopefully they will turn this off by default in a future version of Firefox. I realize that this feature can currently be disabled by the user, but should we really expect all mainstream Firefox users to have to do this to protect their online privacy? I think that is asking too much. IMHO Google has always been a do-no-evil kind of company. Now I feel that they are abusing my trust by sending my traffic to sites that I didn't go to.

While the prefetch issue is clearly bigger than Google, having a top tier internet site like Google endorsing it like this sends the wrong signal to other webmasters. They should remove this "feature" from Google, and Mozilla should disable prefetch by default in future versions of Firefox.

Ben_Graham




msg:904149
 7:52 pm on Mar 31, 2005 (gmt 0)

I'm interested by this statement:
Google only inserts this tag when there is a high likelihood that the user will click on the top result, but clearly this heuristic is not right 100% of the time.

Does this give us insight into human behavior? Can we now tell whether or not people have been clicking on the number 1 result, based on whether or not the prefetch is offered up? There's a lot of variables, but we might be able to learn how to increase click-thrus on organics.

I don't use FireFox, can anyone tell me how often this is actually coming up?

limitup




msg:904150
 8:57 pm on Mar 31, 2005 (gmt 0)

Please tell me they only pre-fetch organic results, and not my premium adwords listings that costs me $2+ a click?

RonS




msg:904151
 9:55 pm on Mar 31, 2005 (gmt 0)

At first blush this seem like it has the potential to make my user/vistitor statistics totally meaningless.

Will they be prefetching Javascript, or images? Will the JavaScript execute if the user doesn't click? Will my tracking show only my image bugs tracking and not the javascript? How will Analog/ AwStats / Webalizer handle this?

Does the Mozilla prefetch send another query if the link is actually clicked (I think this was asked in a different way before, concerning blocking the pre-fetch header. I want to know if another request is sent, prehaps just for the header if the user actually clicks.)

Like I don't have enough to think about at the moment! lol

PCInk




msg:904152
 9:57 pm on Mar 31, 2005 (gmt 0)

Getting all sorts of cookies that I don't want to get. Does anyone else feel that this is a huge violation of online privacy?

I can't find any information on the prefetch that actually explains how it works. Does it simply load the page into the cache without setting any cookies or running any javascript, or does it set cookies/run javascript when the prefetch happens.

If cookies, javascript, ActiveX etc... are not run until the link is clicked, then security should not be too much of a worry.

But I could find no evidence one way or the other?

diddlydazz




msg:904153
 10:21 pm on Mar 31, 2005 (gmt 0)


If cookies, javascript, ActiveX etc... are not run until the link is clicked, then security should not be too much of a worry.

But I could find no evidence one way or the other?

i have been looking for confirmation either way too, but so far nothing.

any members know for sure?

robho




msg:904154
 10:26 pm on Mar 31, 2005 (gmt 0)

Does this mean that there's no way to sort out the difference between a prefetch and a visit after it's made its way into my logs?

With a default Apache combined log, no, there's no difference that I can see (a couple of my sites have the rel="prefetch" tag in the Serps).

However, it does ONLY prefetch the html - no css, images etc. So any stats that show the proportion of users using stylesheets, or browsing with images off, will be skewed (unless a high proportion of the prefetched users do indeed click on the first search result).

Brett_Tabke




msg:904155
 10:30 pm on Mar 31, 2005 (gmt 0)

One wonders what would happen if you prefetched your favorite Google search from your page? hmmm

GeorgePSmith




msg:904156
 10:32 pm on Mar 31, 2005 (gmt 0)

i have been looking for confirmation either way too, but so far nothing.

any members know for sure?

I have run a few tests, and cookies are set and stored during the prefetch.

insight




msg:904157
 10:40 pm on Mar 31, 2005 (gmt 0)

Brett brings up an interesting tangent. Could prefetch be used for DDoS attacks?

Couldn't Google achieve the same effect for IE and other browsers using a hidden iframe? (Not that I'm asking them to do this)

Hanu




msg:904158
 10:47 pm on Mar 31, 2005 (gmt 0)

From reading the Link Prefetching FAQ [mozilla.org]:

If a users does click on a link to a prefetched document, while the prefetch is still in progress, the document will be requested again. That screws your logs even more.

What happens if I click on a link while something is being prefetched?
When the user clicks on a link, or initiates any kind of page load, link prefetching will stop and any prefetch hints will be discarded. If a prefetched document is partially downloaded, then the partial document will still be stored in the cache provided the server sent an "Accept-Ranges: bytes" response header. This header is typically generated by webservers when serving up static content. When the user visits a prefetched document for real, the remaining portion of the document will be fetched using a HTTP byte-range request.

The na´vetÚ behind the next paragraph is even more revolting.

Are there any restrictions on what is prefetched?
Yes, only http:// URLs can be prefetched (https:// URLs are never prefetched for security reasons). Other protocols (such as FTP) do not provide rich enough support for client side caching. In addition to this restriction, URLs with a query string are not prefetched. This is done because such URLs often result in documents that cannot be reused out of the browser's cache, so prefetching them often has little benefit. We found that some existing sites utilize the <link rel="next"> tag with URLs containing query strings to reference the next document in a series of documents. Bugzilla is an example of such a site that does this, and it turns out that the Bugzilla bug reports are not cachable, so prefetching these URLs would nearly double the load on poor Bugzilla! It's easy to imagine other sites being designed like Bugzilla, so we explicitly do not prefetch URLs with query strings. (It might make sense to allow prefetching of these documents when the rel=prefetchrelation type is specified, since this should not appear in any existing content.) There are no other restrictions on the URLs that are prefetched.

Every dynamically generated page has a query string in its URL. Yeah, sure!

Brett, I knew the OR was there for a purpose ;-) Looking at my Ethereal dumps I realized that it isn't active yet on WebmasterWorld - at least I didn't get a 404 for the prefetch. I beg your pardon for screwing up your logs by using Firefox.

diddlydazz




msg:904159
 10:48 pm on Mar 31, 2005 (gmt 0)


I have run a few tests, and cookies are set and stored during the prefetch.

Thanks

I since searched for intel and the SERP resulted in a request from intel .com to set a cookie.

IMO google should think again, this feature wasn't designed for them and they should leave it alone.

The feature itself could be useful but two things i think should be addressed by Mozilla:

quotes taken from: [mozilla.org...]


There is no same-origin restriction for link prefetching. Limiting prefetching to only URLs from the the same server would not offer any increased browser security.

IMHO limiting the prefetch to the SAME domain wouldn't have any effect on its usefulness either.

and the disabling procedure


there is a hidden preference that you can set to disable link prefetching. Add this line to your prefs.js file located in your Mozilla profile directory

but this is a start


By popular demand, Mozilla 1.3+ includes a preference in the UI to disable prefetching. See Preferences->Advanced->Cache to disable prefetching

just my thoughts

Brett, trust you lol

<edit>added quote source</edit>

[edited by: diddlydazz at 10:53 pm (utc) on Mar. 31, 2005]

Liane




msg:904160
 10:52 pm on Mar 31, 2005 (gmt 0)

Why isn't the "opt out" option available on the Firefox - Google preferences page? How is Joe Blow supposed to find out about these things?

Its an invasive feature and I want to opt out! Google is going too far with all this stuff. I don't like it at all!

I have many friends I like a lot. I have business associates I like a lot too. I have a long term admiration for Google ... but that doesn't mean I will allow any of the above to invade my computer without my knowledge or consent!

Out damn spot ... out I say!

<Added>

By popular demand, Mozilla 1.3+ includes a preference in the UI to disable prefetching. See Preferences->Advanced->Cache to disable prefetching

I have not been able to find this option anywhere. Is it different for Mac users?

zCat




msg:904161
 11:09 pm on Mar 31, 2005 (gmt 0)

Yes, I can also confirm that cookies are set. Using the "Ask for each cookie" setting in Mozilla (Edit->Preferences->Privacy and Security->Cookies), it's weird: searching "intel" on www.google.com for produces a "confirm setting cookie" pop-up dialog for a cookie from www.intel.com . (If I didn't know about prefetch I would assume some sort of malware infection or XSS nastiness...).

Some other testing confirms that only the HTML page is fetched; with static pages, the prefetch request fetches the whole page, clicking on the link produces a 304 response.

Sending a 301 or 302 redirect from the prefetched location doesn't seem to have any effect on the original page.

JavaScript is not executed.

zCat




msg:904162
 11:12 pm on Mar 31, 2005 (gmt 0)



By popular demand, Mozilla 1.3+ includes a preference in the UI to disable prefetching. See Preferences->Advanced->Cache to disable prefetching

I have not been able to find this option anywhere. Is it different for Mac users?

It's available in Mozilla but not Firefox (at least not anywhere I can find it).

Namaste




msg:904163
 11:43 pm on Mar 31, 2005 (gmt 0)

please don't use firefox, or any other advanced browser...it violates yr privary...stick to ie and be SAFE

Craven de Kere




msg:904164
 11:47 pm on Mar 31, 2005 (gmt 0)

Brett wrote:

"One wonders what would happen if you prefetched your favorite Google search from your page? hmmm "

I suspect that if enough webmasters did so (i.e. in the footer of their pages) it would cause unwanted traffic to Google's servers that they would be less likely to monetize.

That is, it would do pretty much the same thing that Google is doing to other webmasters but on a different scale.

Liane




msg:904165
 11:53 pm on Mar 31, 2005 (gmt 0)

Right then ... Firefox is gone and I'm back to using Safari! Too bad, I was just starting to like Firefox. :(

diddlydazz




msg:904166
 12:14 am on Apr 1, 2005 (gmt 0)


It's available in Mozilla but not Firefox (at least not anywhere I can find it).

I use Netscape Navigator 7.2 (as main browser) which makes it easy to turn off too, and it's based on Mozilla/5.0 (for those who dont know)

shri




msg:904167
 3:16 am on Apr 1, 2005 (gmt 0)

<offtopic>

Jeez .. way too much stuff to deal with on the 1st of April. There goes my weekend on a tangent that I had not planned for.

</offtopic>

jdMorgan




msg:904168
 4:12 am on Apr 1, 2005 (gmt 0)

A link offered by zCat in post #2 of this thread leads to the info on how to disable this link-prefetching:

1) Enter "about:config" in your address bar.
2) Scroll down to the "network.prefetch-next" preference line (or use the filter to locate it).
3) Right-click on that line to select it, and pick "toggle" from the context menu.
4) Verify that the option is now "false".

This method should apply to any recent Mozilla-based browsers that are prefetch-capable.

Jim

plumsauce




msg:904169
 4:16 am on Apr 1, 2005 (gmt 0)

At first blush this seem like it has the potential to make my user/vistitor statistics totally meaningless

they'll be glad to introduce you to urchin

so, i take it then that the 302 problem/issue/myth/confusion has been fixed? i mean, obviously, other more pressing problems were attended to before unloading this bit of fluff on the world?

in view of the recent request by the FBI for the server/phpbb logs of a site in the last week, anyone using firefox to query google ought to consider their searches carefully. your ip is going to show up in server logs even if you never clicked through. do you want to have to educate someone reviewing those logs on the finer points of prefetch? rather, do you want to pay to educate a lawyer on how to explain prefetch to another lawyer?

btw, i wonder how many affiliate accounts google has opened up? say the serp is for something hot in affilate markets, like mortage refinancing? google forces prefetch using it's affiliate link id. affilate program sets cookie. user later types in the url when he gets around to visiting the site. buys something. who do you think gets the affiliate commission?

then again, since affilate managers watch referral logs, an increase in referrals, real or imagined, might encourage the affiliate manager to consider increasing his spend on adwords.

this is all so dotcom'ish

aleksl




msg:904170
 5:26 am on Apr 1, 2005 (gmt 0)

<offtopic>
Brett, I guess you don't need any traffic from Poland...your htaccess disables Szucacz, which is a bot of Polish search engine. Just thought I'd mention this before people start blindly copying your code.
</offtopic>

Multiverse




msg:904171
 9:49 am on Apr 1, 2005 (gmt 0)

This is where open source software often starts to go downhill:

When coders implement stuff nobody asked for.

First, OSS often is great, because it is developed to get some task done. And since this is the only goal, it gets the task done better then commercial software.

But who asked for Link-Prefetching? Nobody. I know a lot of people who use computers. Bloody beginners and full-time professionals. No one runs around asking "Do you know a program to get the Link-Prefetching-Task done?" nobody.

Mozilla had this sudden boost when it released FireFox. A less bloated Mozilla. Hint hint...

GlynMusica




msg:904172
 12:41 pm on Apr 1, 2005 (gmt 0)

Aside from the privacy issues...see Google cache etc...

If someone were paying for impression based ads and had a position 1 in Google against a keyword, the results would be shown and indepdent of whether the position 1 is clicked on then google would pre-fetch the page, using the Mozzila browser agent as a cover, and the destination site would have recorded a unique visitor? All in all, the publisher wins (but looses on bandwidth?), the advertiser doesn't.

If that gets thrown into Adsense equation does that have any implications? Going out on a limb but if efficient ads are set by Google as rate of clicked on vs number of times served what then happens to ad efficiency? Would this not go down?

Can't believe in this Broadband high speed internet days, at least in Western Europe, that this is being done so a 100k webpage (which is considered heavy) can be downloaded quicker. That's just over 1 second on most connections!

:)

zCat




msg:904173
 12:55 pm on Apr 1, 2005 (gmt 0)

If that gets thrown into Adsense equation does that have any implications? Going out on a limb but if efficient ads are set by Google as rate of clicked on vs number of times served what then happens to ad efficiency? Would this not go down?

Only the HTML page is prefetched; no JavaScript or anything else is executed, so the prefetch hit won't count as an AdSense impression.

GeorgePSmith




msg:904174
 1:10 pm on Apr 1, 2005 (gmt 0)

I've got a question for GoogleGuy. You said:

I knew folks here would be skeptical

Well, you were right. This "feature" has apparently angered a lot of people. Look at the activity on this thread alone! I have also seen negative feedback in the press [news.com.com], on a few other mailing lists that I am on, as well as on other prominent web sites like Slashdot [it.slashdot.org]. Some view this as a privacy violation since it can result in unwanted third party cookies for sites that one never visited. Some are concerned that they could get in trouble with their employers for visiting sites that they never went to, since those requests will show up in proxy logs. Others are angry that this inflates their web stats with requests from people who never visited their web sites.

As a result of this "feature", I see people talking about switching from Google to other search engines. I see people talking about switching from Firefox to other browsers. This is clearly not the desired effect of this "feature". I have yet to see much positive feedback about this "feature", have you? From your comment above, you clearly anticipated this kind of reaction. Why did nobody else at Google listen to you?

So my question for GoogleGuy is, what is the official method that all of us skeptics can take to formally request that Google remove this "feature"? Is this something that you can help us with, or should we all just shut up and switch to other search engines and web browsers?

cgrantski




msg:904175
 1:53 pm on Apr 1, 2005 (gmt 0)

Looks like there are two issues - user privacy and traffic statistic inflation. My concern is the second. Mozilla's insertion of a string into the header is useless to people who use logs for tracking. It would be far better if Mozilla would put something into the browser identification string for that particular hit, so it can be filtered during processing. Why the heck didn't they?

claus




msg:904176
 2:04 pm on Apr 1, 2005 (gmt 0)

>> .htaccess

Okay, i think these will work - haven't tested them but i did look up in the Apache specs before writing them, so they should be allright. Sort of. I hope. (It depends on your server settings of course, eg. not everyone has "setenvif" enabled) ;)

This one should ban them:

--------------------------------- 
SetEnvIf X-moz prefetch HAVE_X-moz
Deny from env=HAVE_X-moz
---------------------------------

...and this one should 404 them:

--------------------------------- 
SetEnvIf X-moz prefetch HAVE_X-moz
RewriteCond %{ENV:HAVE_X-moz} !^$
RewriteRule .* some-filename-that-does-not-exist.htm [L]
---------------------------------

..or, with another syntax:

--------------------------------- 
SetEnvIf X-moz prefetch HAVE_X-moz="prefetch"
RewriteCond %{ENV:HAVE_X-moz} prefetch
RewriteRule .* some-filename-that-does-not-exist.htm [L]
---------------------------------

In some server configs you don't need the "ENV:" part, but the Apache docs say that you can't do without it... it's a strange world.

...add the [OR] flag as needed.

References:
1) [httpd.apache.org...]
2) [httpd.apache.org...]
3) [httpd.apache.org...]


Added:
I've usually got a few #1's but i have no use for artificial traffic. Send me real users anytime, that won't bother me :)

This 87 message thread spans 3 pages: < < 87 ( 1 [2] 3 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / WebmasterWorld / Website Analytics - Tracking and Logging
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved