homepage Welcome to WebmasterWorld Guest from 54.234.2.94
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / WebmasterWorld / Website Analytics - Tracking and Logging
Forum Library, Charter, Moderators: Receptional & mademetop

Website Analytics - Tracking and Logging Forum

    
Bot and scraper revenge
RobWiggen




msg:3745681
 2:54 pm on Sep 16, 2008 (gmt 0)

I've been having bot troubles. A site of mine has been exhaustively downloaded by a range of bots, and this is quite annoying because I don't know why (email search, content scrape?).

I'm working on a range of counter measures, and one that I came up with is based on the fact that most of the bots requested gzipped data. (No doubt so they can save their own bandwidth whilst stealing my content faster.

I have a php script that streams a multi-gigabyte decompression bomb to anyone who hits it. What is the best way to link the file, such that it won't get hit by innocent bots or human users?

 

cdcaveman




msg:3747643
 4:42 am on Sep 19, 2008 (gmt 0)

sounds like war!

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / WebmasterWorld / Website Analytics - Tracking and Logging
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved