Rogue referer....

How did this happen

[joshie76]

We had a URL appear in our referer logs this morning that came as a big surprise... (arch enemies website!)

We checked the page and, not unexpectedly, there was no link to us. So where did the referer string come from and why?

[mark_roach]

I see this quite a lot and have always put it down to someone typing your url into their browser. They were probably on the other page at the time.

I am sure that this doesn't happen all the time though, perhaps someone who understands browsers better would be able to explain why sometimes a referal string is sent and sometimes it is not.

[Will]

A referrer is not returned if the visitor types your URL into their address bar.

Referrer information usually comes from either the HTTP_REFERER or HTTP_FROM headers - depending which one is being used by your report, it is possible to forge quite easily. For example, my logs occasionally show referrers such as "++---------------------".

If there is definitely no link on the other page, my best guess would be that one of the headers has been modified by a server-side script running on your competitor's site.

Why they would want to do that is another matter altogether!

[mark_roach]

More info on sending referral strings [webmasterworld.com].

A referrer is not returned if the visitor types your URL into their address bar.

Perhaps a referrer could be returned if there was a bug with the browser. I just see this too often, and from obscure sites, for it to always be someone faking the referer.

Joshie what was the browser in your referral string ?

[bird]

Various versions of IE have a bug that will return a bogus referrer once in a while. I never cared to check the details, but the most likely possibility seems to be that if you drag and drop a link into a second window, then it could send the URL previously displayed in that window, instead of the one from the window where you dragged the link from (NS sends no referrer in this case, btw.).

[Brett_Tabke]

>A referrer is not returned if the
>visitor types your URL into their address bar.

Some browsers will return a referrer if a url is typed. We get a few dozen referrers here a day from places like NewsHub, Cnet, Wired, ZdNet. We've collectively tried to figure out what it is that triggers a referer like that. It has something to do with the interaction between the back button and caching under IE 4-5. I think IE6 has it fixed.

As mark mentioned, there are also browsers with known referrer bugs. Currently Opera 5 has a pretty serious one in that urls clicked into new windows doesn't always return the referrer, and it can also return the previous root domain instead of the page you visited.

It's been my experience, you can throw out 10-25% of your referrs as bogus depending on the traffic origin.

[my_wan]

Referer or any other headers can be swapped by any user that downloads freeware called "Promoxitron". Basically a proxy server, it can also rewrite parts of any webpage on the fly before it is loaded. Nice piece of freeware.
Ps,
Right clicking and opening link in new window returns no refferer either.

[joshie76]

The UA was Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)

Which looks like a fairly standard IE5

We have even seen referrals from images (.jpg, .gif - I know!!!:)) in the past which, naturally, we've put down to buggy browsers.

I guess, as brett said, you just can't rely on your logs.

[WebGuerrilla]

I had sdomeone call me yesterday who was upset over the fact that he found a referral in his logs from my robots.txt file. He thought I was up to something devious.

The only explanation I could come up with was someone with a buggy browser typing in his URL after viewing my robots.txt file.