Forum Moderators: phranque
How can I force an agent to reveal itself before allowing it access?
If it is able to access your site, then something is wrong with your access-control code, or your expectations.
If you are denying by blank user-agent, then blank user-agents should be denied anytime, all the time, and instantly.
The HTTP User-agent header is sent by the client with each request to your server. The server (for example, your .htaccess code) can examine it and act before the content-handler phase is activated. Therefore, there is no way to 'sneak in' -- it's black and white... allowed or denied.
So I really don't understand the problem. It's best to post your relevant code, post your log entries (after anonymizing specifically-identifiable information) and then ask a question. Otherwise, all we can do is guess, which is not always the best use of time...
Jim
The sample log is me turning off my nightly build's UA and still being able to access pages. The only way I can get the 403 is if I hit the go button?
x.x.x.x - - [09/Aug/2005:15:30:35] "GET /home/home-news.php HTTP/1.1" 200 32615 "http://www.example.com/home/home-news.php" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8b4) Gecko/20050722 Firefox/1.0+"
x.x.x.x - - [09/Aug/2005:15:30:35] "GET /themes/css-theme-classic-du.css HTTP/1.1" 304 - "http://www.example.com/home/home-news.php" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8b4) Gecko/20050722 Firefox/1.0+"
x.x.x.x - - [09/Aug/2005:15:30:41] "GET /home/home-music.php HTTP/1.1" 200 15772 "http://www.example.com/home/home-news.php" "-"
x.x.x.x - - [09/Aug/2005:15:30:41] "GET /themes/css-theme-classic-du.css HTTP/1.1" 304 - "http://www.example.com/home/home-music.php" "-"
x.x.x.x - - [09/Aug/2005:15:30:41] "GET /images/interface-current.gif HTTP/1.1" 200 882 "http://www.example.com/home/home-music.php" "-"
x.x.x.x - - [09/Aug/2005:15:30:42] "GET /home/home-music.php HTTP/1.1" 200 15772 "http://www.example.com/home/home-music.php" "-"
x.x.x.x - - [09/Aug/2005:15:30:42] "GET /themes/css-theme-classic-du.css HTTP/1.1" 304 - "http://www.example.com/home/home-music.php" "-"
Here is the code...
# BLOCK *Faked* blank referer -OR- UA (malicious agents supply a literal hyphen as UA string)
RewriteCond %{HTTP_REFERER}<->%{HTTP_USER_AGENT} ^-<->¦<->-$
RewriteRule \.(htm?l¦php¦txt¦css¦js)$ - [F]
#
# BLOCK blank referer -AND- UA except for HEAD and favicon requests
RewriteCond $1!^favicon\.ico$
RewriteCond %{REQUEST_METHOD}!^HEAD$
RewriteCond %{HTTP_REFERER}<->%{HTTP_USER_AGENT} ^<->$
RewriteRule \.(htm?l¦php¦txt¦css¦js)$ error/error-403-ua.php [L]
[edited by: jdMorgan at 10:17 pm (utc) on Aug. 9, 2005]
[edit reason] Examplified. [/edit]
ErrorDocument 403 /error/error-403-ua.php
#
# BLOCK *Faked* blank referer -OR- UA (malicious agents supply a literal hyphen as UA string)
RewriteCond %{HTTP_REFERER}<->%{HTTP_USER_AGENT} ^-<->¦<->-$
RewriteRule \.(htm?l¦php¦txt¦css¦js)$ - [F]
#
# BLOCK blank referer -AND- UA except for HEAD and favicon requests
RewriteCond $1 !^favicon\.ico$
RewriteCond %{REQUEST_METHOD} !^HEAD$
RewriteCond %{HTTP_REFERER}<->%{HTTP_USER_AGENT} ^<->$
RewriteRule \.(htm?l¦php¦txt¦css¦js)[b]$ - [F][/b]
Replace all broken pipe "¦" characters above with solid pipes from your keyboard before use.
[added] Since you are providing a referer, neither of the code sections above will block your accesses. The first requires either the referrer or the user-agent to be a literal "-", which is a rare but nasty exploit. The second requires both the UA and the referrer to be blank in order to block, except for the noted exceptions.
When you type an address into your address bar and click "go", then there is no referrer, which is why that is working -- If both the referrer and UA are blank, it blocks, as documented. [/added]
Jim
%{HTTP_REFERER}<->
...but it is still allowing blank useragents. I want to block ALL blank useragents whether they have a referer or not, so why after removing reference to any referer am I not able to block?
# BLOCK blank UA except for HEAD and favicon requests
RewriteCond $1!^favicon\.ico$
RewriteCond %{REQUEST_METHOD} !^HEAD$
RewriteCond %{HTTP_USER_AGENT} ^$
RewriteRule \.(htm?l¦php¦txt¦css¦js)$ - [F]
If your site is popular and "public" I suggest that you do not use this approach. A cookies-based method is much better. Set a cookie from any page authorized to link to your private content, then use mod_rewrite or a script (better) to check the cookie before allowing access to the page. If you use a script to both generate and check cookies, you can use checksumming and other methods to allow you to detect forged cookies. You can also check the expiry date and other parameters in addition to what mod_rewrite can check.
Jim
