Protecting your content

took a while but now i have it cracked

[Chris999]

I've noticed that content from my site is frequently copied by others for their AdSense sites. Teleport Pro is the tool that many MFA sites use to steal my content.

First I tried blocking IP addresses but found this to be useless. If you block one IP address they just switch or use a proxy server.

The solution is to convert your html files to .shtml files. The first line of the .shtml file is an include of a PHP IP address counter script.

It only allows 100 page views per IP address each day. Obviously this causes problems for a small minority of users but I have used it for 12 months and no one steals my content now.

Just thought I'd share this.

[ncreegan]

Why max out the number of pages views? I'd imagine a more effective solution would be to block traffic by viewing profile. A la, you're not gbot or slurp, and you're requesting a new page every 3 seconds, so you get banned.

[albl]

if you use apache look at mod_evasive - it does just this

[Chris999]

You can't block by User-Agent because they can easily be faked. Also, IIS only has basic security.

I'm interested in alternative solutions for IIS.

[Edge]

Chris999, can you sticky me the technical details on how you do this? The scripts invovled?

[ncreegan]

I wasn't suggesting blocking by UA -- it's easy enough to get your hands on a IP whitelist.

[topr8]

>>I'm interested in alternative solutions for IIS.

i block by user agent using the global.asa ... session start
people can fake ua stings that is for sure but i also block by ip in the same way.

connect to a database in the global.asa check the UA against the 'blacklist' also check the ip against the blacklist.

populate the blacklist however you like

[leadegroot]

I'm waiting on IncrediBill to release his script ;)

[danimal]

i stumbled across incredibill's site a few weeks ago, it's a riot! i was also unable to find that mythical script :-)

[leadegroot]

There is a script somewhere in the WebmasterWorld archives that lets you detect how long and how often a visitor is hitting and ban them for a day when they exceed a threshold. JFGI

[jstar]

You can't block by User-Agent because they can easily be faked.

I don't think it hurts to include User-Agent blocking, to weed out the low hanging fruit of theives that are too dumb to mask their Wget or whatever tool they're using to download an entire site, and keep them from stealing those 100 pages a day under your IP count method.

[Leonard0]

The script for banning bots (written by by xclus) is located in the PHP forum at [webmasterworld.com...] and also [webmasterworld.com...]

It bans an IP if the number of accesses exceeds a certain amount over a specified time limit.
To speed up the process IPs are stored in a 2 to 4 digit hash so it may ban a few sites accidentally, especially on a busy site.

[kingkong]

I always liked the idea of the hidden/ 1x1 pixel used to trigger a blocking script. Realistically its only going to be picked up by a bot. You'll still need have to differentiate between good bots like google etc and the rest but its certainly helped me fend off a lot of the bad guys.