Awstats analysis

Forum Moderators: DixonJones

Message Too Old, No Replies

Awstats analysis

awstats, htaccess

marodhum

9:50 pm on Dec 21, 2007 (gmt 0)

Hi,
i am using awstats for my logfile analysis. In my reports - visitors by country, i am seeing some visitors from russian union (ru), though in my .htacces file i have Deny from .ru ;)
Is my awstats are wrong?..please help.
Ooh! i forgot to say, i am using apache server.

Also if anybody can help me to stop viewing of my awstats report, publicly, it will be appreciated.

coopster

2:10 am on Dec 27, 2007 (gmt 0)

We might need to see your full Deny Apache directive
[httpd.apache.org...]

As far as keeping the stats from public viewing, at bare minimum you should use a per-directory override file (.htaccess) to keep folks out of that directory!
[httpd.apache.org...]

marodhum

3:36 pm on Dec 27, 2007 (gmt 0)

Hi coopster,
thx for replying. here is my .htaccess


<IfModule mod_rewrite.c> 
RewriteEngine On 
RewriteBase / 
RewriteCond %{HTTP_HOST}!^www\.example\.com [NC] 
RewriteRule ^(.*) http://www.example.com/$1 [L,R=301] 
</IfModule> <limit GET> 
SetEnvIfNoCase user-agent "^Yahoo-MMCrawler" bad_bot 
SetEnvIfNoCase user-agent ^$ bad_bot 
SetEnvIfNoCase user-agent "indy library" bad_bot 
SetEnvIfNoCase user-agent "charlotte" bad_bot 
SetEnvIfNoCase user-agent "snapbot" bad_bot 
SetEnvIfNoCase user-agent "gigamega.bot" bad_bot 
order allow,Deny 
Deny from 38.0.0.0/8 
Deny from 58.22 
Deny from 58.72 
Deny from 58.73 
Deny from 58.74 
Deny from 58.75 
Deny from 58.76 
Deny from 58.77 
Deny from 58.78 
Deny from 58.79 
Deny from 64.34 
Deny from 64.239.0.0/17 
Deny from 66.186.60.44 
Deny from 66.17.15.154 
Deny from 66.17.15.128/26 
Deny from 66.79.160.0/19 
Deny from 66.232.96.0/19 
Deny from 67.15.0.0/16 
Deny from 69.64.73.123 
Deny from 70.87.6.117 
Deny from 72.21.32.0/19 
Deny from 72.29.68.128 
Deny from 72.36.128.0/17 
Deny from 72.232.0.0/16 
Deny from 75.126.105.37 
Deny from 81.177 
Deny from 82.103 
Deny from 84.88 
Deny from 87.218 
Deny from 87.219 
Deny from 125.250 
Deny from 144.140 
Deny from 147.243 
Deny from 189.142.161.83 
Deny from 192.129.3 
Deny from 195.13.172 
Deny from 195.49 
Deny from 202.99 
Deny from 203.99.204.14 
Deny from 207.44.132 
Deny from 207.46.18.30 
Deny from 207.234.128.0/17 
Deny from 208.0.0.0/11 
Deny from 208.32.0.0/14 
Deny from 209.167 
Deny from 212.106.192.74 
Deny from 213.230.128 
Deny from 213.230.129 
Deny from 213.230.130 
Deny from 213.230.131 
Deny from 213.230.132 
Deny from 216.32.0.0/14 
Deny from 216.255.176.0/20 
Deny from 216.67.229.190 
Deny from 216.94 
Deny from 216.95 
Deny from 217.198.147.6 
Deny from 218.30.84 
Deny from 219.128 
Deny from 219.129 
Deny from 219.130 
Deny from 219.131 
Deny from 219.132 
Deny from 219.133 
Deny from 219.134 
Deny from 219.135 
Deny from 219.136 
Deny from 219.137 
Deny from 221.232 
Deny from 221.233 
Deny from 221.234 
Deny from 221.235 
Deny from 222.64.140 
Deny from 222.64.141 
Deny from 222.64.142 
Deny from 222.64.143 
Deny from 222.184 
Deny from 222.185 
Deny from 222.186 
Deny from 222.187 
Deny from 222.188 
Deny from 222.189 
Deny from 222.190 
Deny from 222.191 
Deny from 222.192 
Deny from 222.193 
Deny from 222.194 
Deny from 222.195 
Deny from .de 
Deny from .ru 
Deny from Theplanet.com 
Deny from biz.arrival.net 
Deny from auna.net 
Deny from layeredtech.com 
Deny from lw.seventwentyfour.com 
Deny from panscient.com 
Deny from primarydns.com 
Deny from rf.luispnmd.com 
Deny from svabuse.info 
allow from all 
Deny from env=bad_bot 
</Limit> 
Options +FollowSymLinks 
RewriteEngine on 
RewriteCond %{HTTP_USER_AGENT} ^page_prefetcher [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^Wget [NC,OR]  
RewriteCond %{HTTP_USER_AGENT} ^CherryPickerSE [NC,OR]  
RewriteCond %{HTTP_USER_AGENT} ^CherryPickerElite [NC,OR]  
RewriteCond %{HTTP_USER_AGENT} ^EmailCollector [NC,OR]  
RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [NC,OR]  
RewriteCond %{HTTP_USER_AGENT} ^WinHttpRequest.5 [NC,OR]  
RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^IRLbot [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^zeus [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^webpix [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^libwww-perl [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^(Python.urllib�Java/?[1-9]\.[0-9]) [NC,OR] 
RewriteCond %{HTTP_USER_AGENT} ^pingdom [NC] 
RewriteRule ^.*$ X.html [F,L]

Regarding protecting .htaccess, it is in my root folder and i'm on a shared server.
my host has informed me, if someone directly ask for .htaccess, they will eat 403. i mean if someone ask for http://www.example.com/.htaccess, they will be denied access.
is not that correct? i am a newbie webmaster, so any help will be appreciated.

coopster

4:04 pm on Dec 27, 2007 (gmt 0)

You misunderstood that part. Yes, by all means your server should not allow anybody to view per-directory override (.htaccess) files and any hosting provider worth their salt will have that set up properly on the server. Sounds like you are covered there.

What I meant was to protect the directory/resource that contains your statistical information. For example, if your statistics were in a resource such as http://www.example.com/statistics/ then you would want to have a special override file in that directory which denies access to just anybody (the public) yet allows you to somehow access and view the information. Your host should likely already have this set up for you, if not, ask them how you can do so.

Your Order [httpd.apache.org] directive looks fine there -- didn't need quite that much detail as I was mostly concerned about the order that the deny and allow directives were being evaluated. You can read more about the directives in the link provided and how it all works together. The only other thing I notice is that you have it all wrapped in a GET HTTP method limitation. If your .ru sites are using something other than an HTTP GET (or HEAD) request, then they won't be caught in your filtering process.

marodhum

7:40 pm on Dec 27, 2007 (gmt 0)

Hi mod,
Thanks again for your prompt answer. sorry for being such a dumb
my stats are generating through a perl script which is sitting in my cgi-bin directory. The cgi-bin is protected by default, but when i ask my stats in browser it generates without password prevention. I don't know nothing about script. :(
Guess, have to ask my provider about that.
Regarding this GET HTTP method, i have only one form in my site, and there i have not seen any POST from .ru Will you please tell me or forward me a link, where I will be able to know about other methods.
Btwn... those .ru sites are not mine ;)

marodhum

9:40 pm on Dec 27, 2007 (gmt 0)

I am in the seventh heaven now :)
After much google-ing, i found what i was looking for and solve my problem. There is a file manager in my Cpanel, which allowed me to password protect my log.
Also it will save me from referer spam, which i didn't know earlier.
Thank you again mod for your support and hope to have your support in future.

coopster

1:34 pm on Dec 28, 2007 (gmt 0)

Glad you got it figured out, marodhum! Nice job.