Welcome to WebmasterWorld Guest from 54.158.86.243

Forum Moderators: incrediBILL & martinibuster

Message Too Old, No Replies

Protecting your content

took a while but now i have it cracked

     
1:18 pm on Mar 28, 2006 (gmt 0)

New User

10+ Year Member

joined:Nov 30, 2005
posts:37
votes: 0


I've noticed that content from my site is frequently copied by others for their AdSense sites. Teleport Pro is the tool that many MFA sites use to steal my content.

First I tried blocking IP addresses but found this to be useless. If you block one IP address they just switch or use a proxy server.

The solution is to convert your html files to .shtml files. The first line of the .shtml file is an include of a PHP IP address counter script.

It only allows 100 page views per IP address each day. Obviously this causes problems for a small minority of users but I have used it for 12 months and no one steals my content now.

Just thought I'd share this.

1:23 pm on Mar 28, 2006 (gmt 0)

Full Member

10+ Year Member

joined:Apr 8, 2004
posts:215
votes: 0


Why max out the number of pages views? I'd imagine a more effective solution would be to block traffic by viewing profile. A la, you're not gbot or slurp, and you're requesting a new page every 3 seconds, so you get banned.
1:25 pm on Mar 28, 2006 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 7, 2006
posts:53
votes: 0


if you use apache look at mod_evasive - it does just this
1:33 pm on Mar 28, 2006 (gmt 0)

New User

10+ Year Member

joined:Nov 30, 2005
posts:37
votes: 0


You can't block by User-Agent because they can easily be faked. Also, IIS only has basic security.

I'm interested in alternative solutions for IIS.

1:39 pm on Mar 28, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 29, 2001
posts:1113
votes: 32


Chris999, can you sticky me the technical details on how you do this? The scripts invovled?
2:22 pm on Mar 28, 2006 (gmt 0)

Full Member

10+ Year Member

joined:Apr 8, 2004
posts:215
votes: 0


I wasn't suggesting blocking by UA -- it's easy enough to get your hands on a IP whitelist.
2:34 pm on Mar 28, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member topr8 is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 19, 2002
posts:3269
votes: 20


>>I'm interested in alternative solutions for IIS.

i block by user agent using the global.asa ... session start
people can fake ua stings that is for sure but i also block by ip in the same way.

connect to a database in the global.asa check the UA against the 'blacklist' also check the ip against the blacklist.

populate the blacklist however you like

8:33 pm on Mar 28, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 27, 2003
posts:1648
votes: 2


I'm waiting on IncrediBill to release his script ;)
1:06 am on Mar 29, 2006 (gmt 0)

Senior Member

joined:May 5, 2005
posts:735
votes: 0


i stumbled across incredibill's site a few weeks ago, it's a riot! i was also unable to find that mythical script :-)
1:21 am on Mar 29, 2006 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 27, 2003
posts:1648
votes: 2


There is a script somewhere in the WebmasterWorld archives that lets you detect how long and how often a visitor is hitting and ban them for a day when they exceed a threshold. JFGI
2:11 am on Mar 29, 2006 (gmt 0)

New User

10+ Year Member

joined:Jan 25, 2005
posts:38
votes: 0


You can't block by User-Agent because they can easily be faked.

I don't think it hurts to include User-Agent blocking, to weed out the low hanging fruit of theives that are too dumb to mask their Wget or whatever tool they're using to download an entire site, and keep them from stealing those 100 pages a day under your IP count method.

9:42 pm on Mar 29, 2006 (gmt 0)

Full Member

10+ Year Member

joined:Sept 14, 2005
posts:272
votes: 0


The script for banning bots (written by by xclus) is located in the PHP forum at [webmasterworld.com...] and also [webmasterworld.com...]

It bans an IP if the number of accesses exceeds a certain amount over a specified time limit.
To speed up the process IPs are stored in a 2 to 4 digit hash so it may ban a few sites accidentally, especially on a busy site.

7:43 am on Mar 30, 2006 (gmt 0)

New User

10+ Year Member

joined:Mar 20, 2006
posts:10
votes: 0


I always liked the idea of the hidden/ 1x1 pixel used to trigger a blocking script. Realistically its only going to be picked up by a bot. You'll still need have to differentiate between good bots like google etc and the rest but its certainly helped me fend off a lot of the bad guys.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members