Referrer = "-"

Being Inundated By Mystery Referrer

[CheeseburgerBrown]

My web traffic has been quadruple normal for the past three days. I want to know where all the hits are coming from, but it seems that most of this influx of traffic shows up in my logs with a referrer of "-" (no quotes).

I've been link:mysite/resource-in-question through Google to see if I can find where it's coming from, but the index doesn't seem to have caught up with us yet.

This might be a stupid question, but is there any other way I can try to find out where all of these new hits are coming from?

Any insights appreciated.

[Ultraseeker]

If your web stats reports have this information you might want to look at the browswers and spiders hits for that period. If it doesn't show you spider information, you might want to download your logs and run an analyzer software like Weblog and check for spider activity. I've seen a bunch of spiders that are not coming from Google or Inktomi or any of the major search engines. Some of them claim to search your site to prevent plagiarism... They can really eat up your bandwith though. If you find such bots slamming your site you could try using robots.txt although not everyone of them will honor it.

[CheeseburgerBrown]

Naw, it's not spiders or bots. It human beings -- European ones...mostly German and Austrian.

I just can't figure out why a website (apparently up to nothing nefarious) should want to suppress its own referer information when offering its visitors links to interesting media.

[Sanenet]

Maybe a site in Germany / Austria is stealing your content? If they're retrieving copies of your page via a script, it wouldn't send a referer. Do those pages have any daily changed content?

[Frequent]

Could this be from someone who has their browser configured not to pass referrer info?

Freq----

[bcolflesh]

Lots of software removes/can remove the referrer info - various ad/virus/trojan blocking progs - most proxies can be configured to do this as well.

[CheeseburgerBrown]

I've seen blocked referers on *individuals* before, just never for everyone coming via a certain site. As I mentioned, *all* traffic from this site has the blocked referer field, no matter what ISP/browser combination the surfers are using.

Since I don't serve multimedia out to foreign hosts, they're not hotlinking my media content. They're definitely coming into a section's principal index and then sampling the media presented there, just like any other visitor.

The traffic is trailing off now, though they did manage to use 50% of my month's bandwidth quota in four days.

So, it's no panic situation -- it's just weird, is all. Why would a non-nefarious site not want to turn up in referer logs? Or, if they up to something nefarious, what could it possibly be?

[arrowman]

I just can't figure out why a website (apparently up to nothing nefarious) should want to suppress its own referer information when offering its visitors links to interesting media.

The first question is not why, but how?

A browser, firewall, proxy and the like can filter the referer. But how can a site that links to your content do the same?

[teaperson]

By opening your site in a new window. I recently learned in another forum here that in IE, when a link is opened in a new window, it doesn't send a referrer (at least that is the case when a user right-clicks the link, so I would think it might also be the case when a link opens in a new window).

[CheeseburgerBrown]

"By opening your site in a new window. I recently learned in another forum here that in IE, when a link is opened in a new window, it doesn't send a referrer..."

That's useful information, if true. Can anyone else confirm or deny this, or explain which configuration/circumstances may allow it?

[larryn]

Microsoft is pretty clear about when it does not include referrers in "Internet Explorer Does Not Send Referer Header in Unsecured Situations" - check for Article ID 178066 [support.microsoft.com ]

[DanA]

IE sends a referrer except in the cases mentioned above.
Some browsers using IE may block the referrer as an option.
The referrer would be blank (most of the time) not a "-".

[CheeseburgerBrown]

I just thought of something: the page being visited has been moved, and there's a 301 directive from the old URI. Could it be that these hits are all pointed at the old URI (not indicated in my logs), and then being passed along by the 301 Redirect strips the referer?

Does this theory hold any water?