|How can I foil a hit log?|
My handle says it all.
I've recently discovered that a web site can tell what search terms I used in a search engine to find their web site.
Most times I don't care.
But sometimes I do. I don't like being profiled and tracked. It depends on the web site and the search terms.
Scenario: I enter search terms and get a list of site hits from... say Yahoo.
How do I get to that site without clicking on the link that's given? (I could highlight the text of the link and paste it into the addy bar and edit it and remove the terms, but it's time-consuming.)
I was thumbing through a book today... Google hacks.. there's something called a Google API. Could I write my own search interface that could somehow modify the way the link is returned so that it won't contain the search terms? Does Yahoo use Google technology?
And a related question...
I noticed that even if I type a url in the addy bar, if I'm currently viewing the Yahoo home page let's say, that when I hit enter, Yahoo's url is somehow appended to the url that I typed in (I use Netscape and can see the combined url in the bottom left hand corner) and the whole thing get's sent out as a request. What does that mean to the web server when my request arives?
What info does that convey to the web site? That I was looking at Yahoo obviously. Anything else?
Thanks. I hope this question doesn't mean a conflict of interest between me and webmasters. You just have to admit.. there are some webmasters that are sneaky and unscroupulous, even to other webmasters (as I've seen in some of the posts here)
Get yourself a decent local proxy or buy & install one of the many "internet privacy and secury" packages on the market (eg. Norton or whatever they call themselves these days).
Normally they have a setting that lets you remove / replace the entire referrer header (ie the name of the page that brought you here, which in the case of a search engine referral also includes the search terms), which then stops a website learning about the referring page "passively".
Welcome to WebmasterWorld [webmasterworld.com], privacyfanatic.
Most of what you mention is related to the HTTP referer header sent out by the browser when clicking on links: it gives the URL of the page you came from. You say you are using Netscape, but you don't say which version. If you are using 7.1, then you can turn off the referer (you'll need to search for the exact way to do it, sorry).
Short of using an anonymizing proxy service (and that's not perfect either), you can't ensure complete anonymity: however, you can avoid over-zealous profiling with comparative ease.
Hmmm.. I have HS 7.1 actually. I have to figure it out.
I looked into the proxy thing. (in fact I'm using one now!)
BTW... there's a IE plug-in called proxy-switcher that makes changing proxy's a snap. W/O it you need to drill through all these pages and type it in by hand. So, when I want to use a proxy, I use IE... I noticed (well the proxies I use anyway) that proxies don't always work the entire time I'm online. I guess some proxy servers have a bandwidth limit, so I'm forever switching proxies. But with this tool it's a snap.
Hmmm... well I guess I'm going to look into the 7.1 feature and use it in conjunction with a *web-based* proxy, ie. one that uses a web page to surf from instead of using the proxy field in the browser's settings page. That way I can have my cake and eat it too.
A growing number of sites will block access to images and other resources if you don't pass the Referer when making a request. The referrer information can also be used to highlight, or take a visitor directly to, the most relevant section of a site.
Just so you know ;)
although you're name is a bit of a giveaway, I still have a question (although it is slightly OT):
"I've recently discovered that a web site can tell what search terms I used in a search engine to find their web site. [...] I don't like being profiled and tracked."
Aren't you a bit oversensitive? When I look through my logfiles I'm mostly interested in the search terms used. I don't really see how I could track or profile anyone. I look at the search strings and see what is of interest to my visitors and how I rank in the SE result lists. What I don't see is who the person is and where else he went before or after going to my site.
I'm somneone who is very meticulous about my privacy, but it does not only bother me that webmasters can see my search terms, but I also know that they are of great value to them.
See my post - Question about logging.
How about clicking on the link to the cached page if it is availiable?
Just tried it and it does the same thing.
And also, if you click the cache you end up on a totally different web page (most times I tried it anyway).
I'm going to get an Apache server shortly and then I'll be able to figure it all out.
Various internet privacy products will replace your referer string with a dummy string for you, if you want. I think Norton Internet Security (their firewall product) will do that, if you set it up to.
Excuse me if I'm thinking too simple here, but how about opening a new browser window, simply typing the website URL into that, without any of the search engine stuff in it, just the target website?
I went to Yahoo and searched "Dicky Betts"
I got a SERP.
I tried the following:
Click the link in the SE SERP:
Sends out referer string and a Get string like..
GET /S=2766679/K=dicky+betts/... (Netscape)
GET [rds.yahoo.com...] (IE)
result: I get to the Dicky Betts website.
Right-click on link in SERP and copy link target and paste into browser:
No referer, and same Get strings.
result: Get redirected to some wierd site.
And of course... simply typing in the webpage url.. which again didn't give me the right page.
And I'm curious as to *what* in the referer field is Dicky's webmaster looking at?
I'm tring to build a browser/server using Visual Basic where I can play with this stuff. But I'm not doing something right. Once I figure out what I'm doing wrong I can play with the headers and send out connection requests and see what loads.
As someone said earlier, he looks at the search terms, as do I, personally I use AWStats to parse through the logs, which leaves me with a list of search terms entered in search engines, and logged via the referer header...
Now some people do browse with it turned off, and they probably enjoy not getting to any images on my site, but that's their business. However I mainly want to be able to see the search terms used to find my site, that way I know if there's an increased interest in something I have done (it's basically all little program type things I've done).
Also by logging the referers I can see when sites are linking to my site, and where they are linking to - for instance if some site decided to link to one of my programs I'd be able to go and see exactly what they've written about it...
As you might notice, I'm only interested in the referrer, not the person doing the browsing. To me, that is just a numerical statistic - x people visited my site, and y of them were unique. There will always be people who get paranoid about what is being tracked (normally they also don't understand much about the way the web works) but by far they are in the minority, so my stats, for the forseeable future, are useful.
What a lot of people who like their privacy tend to ignore is the sheer amount o information that is being tracked - an anonymizer service might sound good, but your resolving of a hostname can be logged, if the anonymizer is on a standard port your isp can be logging every page you ask for, the anonymizer might be sending your originator IP address anyway, even if it's not, most anonymizer services are known, and a lot of webmasters don't like that, so block, or send different information to them...
The only safe way to stay anonymous online is to stay offline basically, failing that use a public building with net access (such as a library etc).
When I do a search on Yahoo (just tried one) it shows the url of the target page in green below each result.
I can simply copy & paste this url into a new browser window, and get directly to the correct page, and no referrer or yahoo keywords etc are sent.
Ok... search "Dicky Betts"
Try clicking the link in the SERP. Then try your method.
There's a difference. The second one won't get you to the website.
I guess the bottom line here is: There's times where referer strings are good and harmless and benificial(most times), and there's times where you want to change them. But if you do change them, you may not land on the page you want. At this point I think we're beating a dead horse.
Thanks for all the replies everybody.