homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Google / Google AdSense
Forum Library, Charter, Moderators: incrediBILL & jatar k & martinibuster

Google AdSense Forum

Protecting your content
took a while but now i have it cracked

 1:18 pm on Mar 28, 2006 (gmt 0)

I've noticed that content from my site is frequently copied by others for their AdSense sites. Teleport Pro is the tool that many MFA sites use to steal my content.

First I tried blocking IP addresses but found this to be useless. If you block one IP address they just switch or use a proxy server.

The solution is to convert your html files to .shtml files. The first line of the .shtml file is an include of a PHP IP address counter script.

It only allows 100 page views per IP address each day. Obviously this causes problems for a small minority of users but I have used it for 12 months and no one steals my content now.

Just thought I'd share this.



 1:23 pm on Mar 28, 2006 (gmt 0)

Why max out the number of pages views? I'd imagine a more effective solution would be to block traffic by viewing profile. A la, you're not gbot or slurp, and you're requesting a new page every 3 seconds, so you get banned.


 1:25 pm on Mar 28, 2006 (gmt 0)

if you use apache look at mod_evasive - it does just this


 1:33 pm on Mar 28, 2006 (gmt 0)

You can't block by User-Agent because they can easily be faked. Also, IIS only has basic security.

I'm interested in alternative solutions for IIS.


 1:39 pm on Mar 28, 2006 (gmt 0)

Chris999, can you sticky me the technical details on how you do this? The scripts invovled?


 2:22 pm on Mar 28, 2006 (gmt 0)

I wasn't suggesting blocking by UA -- it's easy enough to get your hands on a IP whitelist.


 2:34 pm on Mar 28, 2006 (gmt 0)

>>I'm interested in alternative solutions for IIS.

i block by user agent using the global.asa ... session start
people can fake ua stings that is for sure but i also block by ip in the same way.

connect to a database in the global.asa check the UA against the 'blacklist' also check the ip against the blacklist.

populate the blacklist however you like


 8:33 pm on Mar 28, 2006 (gmt 0)

I'm waiting on IncrediBill to release his script ;)


 1:06 am on Mar 29, 2006 (gmt 0)

i stumbled across incredibill's site a few weeks ago, it's a riot! i was also unable to find that mythical script :-)


 1:21 am on Mar 29, 2006 (gmt 0)

There is a script somewhere in the WebmasterWorld archives that lets you detect how long and how often a visitor is hitting and ban them for a day when they exceed a threshold. JFGI


 2:11 am on Mar 29, 2006 (gmt 0)

You can't block by User-Agent because they can easily be faked.

I don't think it hurts to include User-Agent blocking, to weed out the low hanging fruit of theives that are too dumb to mask their Wget or whatever tool they're using to download an entire site, and keep them from stealing those 100 pages a day under your IP count method.


 9:42 pm on Mar 29, 2006 (gmt 0)

The script for banning bots (written by by xclus) is located in the PHP forum at [webmasterworld.com...] and also [webmasterworld.com...]

It bans an IP if the number of accesses exceeds a certain amount over a specified time limit.
To speed up the process IPs are stored in a 2 to 4 digit hash so it may ban a few sites accidentally, especially on a busy site.


 7:43 am on Mar 30, 2006 (gmt 0)

I always liked the idea of the hidden/ 1x1 pixel used to trigger a blocking script. Realistically its only going to be picked up by a bot. You'll still need have to differentiate between good bots like google etc and the rest but its certainly helped me fend off a lot of the bad guys.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google AdSense
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved