Deny content to a blank UA?

All methods and suggestions are welcomed!

[JAB Creations]

I am curious if there is a way to negate sending clientside code to a request by a user with a blank or dash useragent? Also would there be a way to enact an Apache code such as 403 if the client failed this requirement?

Apache is failing to 403 except on direct requests and there has been no working example of a work around for this. Therefor it is my new goal to use all serverside methods of denying content to spammers. Clientside code is obviously useless as the content is already served.

I welcome any suggestions!

[DanA]

While an empty UA doesn't mean a spammer,
$UA=getenv("HTTP_USER_AGENT");
if ($UA==""){die();}
will deny content
You can insert a header with any status before exiting or echo a fake html page.

[Eltiti]

Do you only want to serve content to scrapers etc. that spoof a particular UA?! ;-)

[JAB Creations]

Thanks, this works 100% effectively!

$UA=getenv("HTTP_USER_AGENT");
if ($UA==""){header("HTTP/1.0 403 Forbidden");die();}

Eltiti, not sure what you mean?

[Longhaired Genius]

I think DanA only meant that code to be an illustration. It may not be a good idea to ban all requests without a user-agent string. Many web-caches, including those of AOL, do not provide a user-agent string.

[JAB Creations]

Knock knock...

Who's there?

...

I thought I heard a knock at the door? Oh well, no reason to open it.

If someone knocks on my door and does not answer when I ask who's there then why should I bother to open the door?

100% of the ip addresses looked up (between a 4 to 20 minute delay typically) are listed in at least one spam list. Overwhelmingly most ips are listed in several.

I appreciate the heads up on the aol cache servers and such. I'll be sure to give them a phone call and let them know to get a new IT dept.

[Eltiti]

not sure what you mean

I meant that it is trivial to "spoof" (i.e., pretend to be) any user agent, so I never try to "protect" content by looking at the UA. (Of course you could say that, while the *presence* of a UA string is not enough reason to *give* access to the content, its *absence* could be a reason to *deny* access...)

[jatar_k]

this easy

[php.net...]

CURLOPT_USERAGENT
The contents of the "User-Agent: " header to be used in a HTTP request.

I usually use my own exact user agent when using curl. I am almost of the mind that no user agent has a higher possibility of being legit than non.

[DanA]

I would't ban an empty UA, I think that random UAs are more often looking for email addresses.
I use a redirection (with a variable that I test) for those UAs trying to grab pages with the words email, mailto, @...
Usually robots do not follow the redirection and exit with no content.
Then I ask for a confimation.
Usually, robots do not confirm...

[JAB Creations]

1.) If legit agents fail to declare a proper and easily identifiable then they must then be identified via the IP or IP range. Please specify such ranges or ways that I can understand to detect these agents.

2.) Illegitimate agents are accessing my site. My site is not an American border, I intend to adapt and seal security breaches at all possible levels.

3.) Illegitimate agents will always be detectable regardless of how they identify themselves. Adaptation to abuse is the only way to learn how to deal and ultimately stop it. I would rather rather learn lessons by experience in this period of my life now then to deal with the consequences in a professional scenario. There are three general methods of detecting abusive agents and being an idea person I believe there to be more beyond the three I have in mind.

4.) I love a dam good challenge and the web provides a series of steadily greater and greater challenges to be effectively conquered and eventually to be profited from while which I enjoy the work and rewards of accomplishment, plus rent ... that one is always a good point!

Regardless I have seen plenty of evidence of forged UAs and find myself in agreement to one extent or another with what I percieve you are saying. However I'm not quite sure what you are suggesting? I can understand some benefits of using a unique UA.

[JAB Creations]

Dana, I should not micromanage on behalf of people with bad intent.

[5x54u]

JAB

I had the same scenario occur.

IP came to my site every half hour on the hour passing no UA . No info on said IP in ARIn and so I did as you did then noticed a new ip doing same thing. No info from ARIN on that one either.

Then my host emailed me asking if I was checking ids at the door.

Turns out my host uses a third party to check up on all servers and these IPs were from that checking service. He emailed the party to get a response as to why they do not pass UA info and I will post back to you on that response.

[JAB Creations]

Thanks 5x54u! :)