Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Firefox Referrals from Google Search

         

dusky

4:15 am on May 12, 2010 (gmt 0)

10+ Year Member




System: The following 8 messages were cut out of thread at: http://www.webmasterworld.com/google/4129694.htm [webmasterworld.com] by engine - 11:02 am on May 12, 2010 (utc +1)


Talking about referrals, read on, either be inspired or call me the fool who never knew!

I am puzzled by this, using FF, you search G.com for a keyword you know your site is the number 1, say a news article page, now you get the search results, you see that your site is on top with 9 other sites / pages, you close the browser (FireFox). Then the mystery, your are told you had a unique visitor by you stats program on Cpa*al's latest visitors for example and even your site referrer log shows you had that visitor with the G* complete URL containing what you have just searched for, it was your IP address and you NEVER clicked to visit the site.

I tried this about 30-40 times on FF, each time the site is number one, I never click the page to go to the site, YET that search string is in my logs as a unique visitor with my IP.

If I did not spot this, I would think someone visited the site, furthermore, I'd also think that keyword is bringing in traffic when they are probably and very likely pinging every site you are looking at on the SERPs, caching them first in case you click any of them. If this is what's happening, it is not fair on Bing or Yahoo to make me think I had x0000s visitors today INSTEAD of saying your site appeared x0000 times when people searched for that keyword, some might have clicked and probably most did not click.

I disabled firebug and any other add-on / extension including the G* toolbar, always logged out from my G* account, cleared all cache, even cookies but still get this on FF.

I tried on IE, this does not happen, is G* doing the pinging from FF's Built in G* search tools, is it a persistent feature within the toolbar you can't uninstall even if you uninstall the toolbar, or am I years behind. I can understand if the pageviews feature may be enabled by default whether you like it or not, but if it was I should have the same problem on IE.

If everyone thinks the same as me and it's something new, could we have this on its own thread. Surely this has huge implications, imagine you have 1000 clickthoughs one hour from G* search results and you don't know how many are real people and how many are ONLY pings because your page was returned on the search results. They are definitely searches I made myself, no one else could possibly be doing that with my IP address at the precise time, with my exact user agent string from my machine...

Assuming I have a spyware on this machine, surely this action does not make any sense unless the spyware itself is a search engine from which I downloaded a tool or have the ware built into FF.

It can't be to do with PS, I never clicked on that keyword to come to my own site and very rarely do click to the site from G*, I am like the majority here, you search for keywords to monitor SERPs positions!

dusky

4:35 am on May 12, 2010 (gmt 0)

10+ Year Member



To confirm, this happens only if the keyword you search for brings the page / site as the Number one, other positions do not seem to register as click throughs. I don't know it may well be happening on IE and other browsers, but is does not because there is no G* toolbar on my IE. Off to test on all browsers now!

graeme_p

6:41 am on May 12, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Does not happen when I tested it (FF 3.6.3 on Linux).

dusky

6:45 am on May 12, 2010 (gmt 0)

10+ Year Member



Yep, only on FF, all others seem not to cause this (tried on Opera, IE and Safari). Hope I am wrong though and only have a spyware hidden I could not get rid of after few scans with different products plus two online scans, just for the sake of making sure. Stats and traffic count can't be relied up on anymore from G* from people using FF at least, then again how could one know definitely which browser that referral is coming from, this is going to be a tough one to crack indeed.

Note, other browsers have only the search box which includes G* search, no toolbar, whereas on FF, I always had the toolbar itself with PR and all that, even though I disabled it on FF, still happens. Does opting to share info with G* because of the PR means this is also taking liberties?

dusky

6:49 am on May 12, 2010 (gmt 0)

10+ Year Member



Using windows XP SP3 by the way on this machine, I'll reboot into linux and see how things are on FF, however, to reproduce the problem, I have to have the G* toolbar installed on FF and Linux as well.

g1smd

7:05 am on May 12, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Is this simply the page PreFetch facility in action?

dusky

7:50 am on May 12, 2010 (gmt 0)

10+ Year Member



Is this simply the page PreFetch facility in action?
Yes g1smd

I rebooted into Linux and did the search, checked the logs and the so called visitor hit was there. I looked on the G* search result page source and seen <link rel=prefetch href=.... Binged what it was and found this link (http://www.xml.com/lpt/a/1627) which explains it nicely and others about G* using the prefetch tag. So it's NOT FF to blame, but G* which uses it on their page source like I suspected all along to prefetch the number one result's page thinking likely to be clicked next to save on speed, bingo!

This is outrageous, I have to find a way to block this uncalled for, unwanted and un-opted for behavior! sorry it needed to be prominent what I thought to say, I am sure a lot would agree!

g1smd, while I was preparing to post my findings, you must have posted your answer, well at least I know what is causing this!

dusky

7:56 am on May 12, 2010 (gmt 0)

10+ Year Member



Sorry wrong link I posted, it's [lwn.net...]

aristotle

10:18 am on May 12, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Are you sure that it only happens with the Firefox browser? Why would Google do a prefetch when people search with Firefox, but not do a prefetch for searches with other browsers? Or maybe I'm not understanding what you're seeing.

jdMorgan

1:07 pm on May 12, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Client-side: Disable prefetching in Firefox. Type the URL "about:config" and then search for "prefetch" and double-click the network.prefetch-next;true to set it to "false."

Server-side: Check the HTTP X-moz request header and force a 403-Forbidden or 404-Not Found response or serve an alternate file if the header value is set to "prefetch". For example, in Apache /.htaccess:

RewriteCond %{HTTP:X-moz} =prefetch
RewriteRule ^ /no-prefetch.html [L]

This code is a trivial example; you may wish to respond in different ways when various different kinds of resources are requested.

Jim

dusky

2:50 pm on May 12, 2010 (gmt 0)

10+ Year Member



Thanks jdMorgan, someone PMed me a thread link here [webmasterworld.com...] about the prefetch tag and FF, another interesting related discussion, BUT it is browser specific thread in which G* and traffic / SERPs are irrelevant. Unique visitor count is the problem discussed here.

OK, maybe only 30% +- of browser users worldwide come from FF, probably higher in some countries than others, out of that, 1-2% of them come because you ranked number one and they bypassed the link / result and clicked on another result, still those are significant margin of errors. When you think your 1 million monthly unique visitors are actually less those pages you ranked number one and no one clicked to come, yet it registered as a unique visit. FF and G* are deceiving you into thinking some of your uniques are bogus prefetch hits, no one came to your site.

And the above is my argument, traffic stats, some keyword views counted as clicks and real users.

tedster

4:37 pm on May 12, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



those are significant margin of errors

Agreed, this is one of many challenges with website analytics. As long as pr-fetch is the default behavior in Firefox, you do need to consider this factor and not get too excited about a very high CTR. For a more accurate picture, it needs to be adjusted downward, and your conversions will then look a little better than they seem to be, too.

How many site owners even look at visitors by browser and think to make that adjustment for FF pre-fetch behavior? And if you use FF to check your own keywords you certainly do want to turn off pre-fetch on your own browser.

dusky

5:23 pm on May 12, 2010 (gmt 0)

10+ Year Member



I guess checking one's own keywords on a different browser is no problem in eradicating registering "SERP Views" as real clicks when the site is number one for a particular keyword/phrase. For other users, yes one has to allow for a margin of error, who knows the percentage is the problem. Imagine if 100% of users worldwide used FF, now this means at least no less than 10% of "Premium" keyword organic traffic could be bogus (in an ad-hoc situation), assuming 1-10 positions only, not counting people going to page 2 etc. Hard to put the blame on anyone as being 100% culpable, browsers, SEs or other sites, stats software and developers, webmasters themselves for not putting a block on this practice....

Nonetheless, it's hard to stomach that 10,000 people using FF searched for "widgeting widgets indoors" when your site comes first to know that all of them, the whole 10k registered as visitors that day, when probably only 2,4, or 5k actually clicked to come to your site and the rest chose position 2-1000 placed sites due to irrelevance or just did not like the snippet etc. Furthermore, if one is to buy your particular site ignoring this fact, that would be unintended misleading a buyer if the site is mainly assessed by its real unique visitors...an interesting scenario!