| 6:47 pm on Mar 9, 2007 (gmt 0)|
It is doubtfull they are actually sending you traffic.
Instead, they are sending you robots to spam your logs with referrals.
| 8:45 pm on Mar 9, 2007 (gmt 0)|
That's right. Some websites publish their referrals, leading to inbound links to the spammer. If you can determine the IPnr, then you can block access.
| 10:04 pm on Mar 9, 2007 (gmt 0)|
Interesing, so what i need to do then, is examine my raw logs to see what the incoming IP is of these bots, then block them with an entry or two in my Isapi_Rewrite file? (Site runs on windos server). Thanks for the help guys, I had not considered that the traffic was coming from bots or the motive.
| 1:16 pm on Mar 10, 2007 (gmt 0)|
|Instead, they are sending you robots to spam your logs with referrals. |
I so much hate these things, makes log analysis such a trouble sometimes.
Is there a good listing of all known sites generating such fake activities somewhere? So I can filter them all?
| 1:52 pm on Mar 10, 2007 (gmt 0)|
>>Instead, they are sending you robots to spam your logs with referrals.
This is just another example of how the web has turned into a "war zone" with people just wasting bandwidth. Other webmasters throwing junk on your site in the hope - because their is no guarantee - that you actually publish your weblogs.
| 2:42 pm on Mar 10, 2007 (gmt 0)|
Yup, sounds like referral spam. IP blocking and domain blocking will need to be done, i'm not sure where or when they get urls to 'referer spam' i get it on one of my sites, but not to a massive degree, I also have a system which shows for eatch article which articles are refering to it. But thankfully the same system has a very good spam blocking algo :)
| 3:47 pm on Mar 10, 2007 (gmt 0)|
|...examine my raw logs to see what the incoming IP is of these bots, then block them with an entry or two in my Isapi_Rewrite file... |
I use a simple cgi script called "Robot Rat" to capture the ip numbers of any bots crawling my sites. I then block the ip for any of these rodents that are not directly associated with Google, Yahoo, MSN, or one of the minor engines (Ask/Teoma, GigaBlast, Alexa, et al). Thus, I avoid having to study the daily logs as the ip's for the crawlers are neatly stacked each day, one on top of the other. I've cut my wasted bandwidth significantly.
Also, some control panels (cPanel X for example) allow you to easily block ip's so you don't have to re-write anything -- they do it for you after you enter the IP to be banned. If your server is using cPanel, look for the "IP Deny" icon.
| 5:19 pm on Mar 10, 2007 (gmt 0)|
If you use Awstats, then you can use the SkipHosts field in Awstats configuration file. If the spammer shows up as a domain, then simply put :
Skiphosts="google.com yahoo.com mattcutts.com live.com" etc.
If the spammer shows up in various URL under the same domain, then you can work with regular expressions.
Then the spammer will still draw upon your server resources, but not show up in Awstats.
[edited by: Martin40 at 5:20 pm (utc) on Mar. 10, 2007]
| 7:22 pm on Mar 10, 2007 (gmt 0)|
|Skiphosts="google.com yahoo.com mattcutts.com live.com" etc. |
omit "etc." :-)
Skiphosts="google.com yahoo.com mattcutts.com live.com"