Page is a not externally linkable
Natashka - 12:16 am on Oct 8, 2002 (gmt 0)
I don't understand something here, maybe somebody could clear things up for me.
I have a htaccess on my site, just like the one you are discussing here, but it doesn't work for a simple reason: most of the offline browsers don't identify themselves as such! My logs show that all user-agent are either MSIE, or Netscape, or AOL! However, my site is constantly downloaded by those grabbers, some days ago there were 40,000 (!) hits coming from the same IP in one hour period. It looks like that browser got stuck on one page or something. Even my webhost said it was probably offline browser, but all logs indicated that it was MSIE. Yesterday I've even downloaded WebZIP myself, turned off MSIE, entered my site with WebZIP and... logs showed I was using MSIE, and not WebZIP! What's the point of this htaccess if offline browsers "mask" themselves?
I am really frustrated, because not only it sucks up all my bandwidth, but my CTR rate is dropping dramatically, I am afraid my advertisers will just kick me out. Maybe somebody can help and explain me what can be done? I heard there is a way to restrict amount of hits coming from the same IP, anybody knows how to do it? It may be more efficient, since offline browsers "pretend" to be MSIE.