homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / WebmasterWorld / Website Analytics - Tracking and Logging
Forum Library, Charter, Moderators: Receptional & mademetop

Website Analytics - Tracking and Logging Forum

Bot and scraper revenge

 2:54 pm on Sep 16, 2008 (gmt 0)

I've been having bot troubles. A site of mine has been exhaustively downloaded by a range of bots, and this is quite annoying because I don't know why (email search, content scrape?).

I'm working on a range of counter measures, and one that I came up with is based on the fact that most of the bots requested gzipped data. (No doubt so they can save their own bandwidth whilst stealing my content faster.

I have a php script that streams a multi-gigabyte decompression bomb to anyone who hits it. What is the best way to link the file, such that it won't get hit by innocent bots or human users?



 4:42 am on Sep 19, 2008 (gmt 0)

sounds like war!

Global Options:
 top home search open messages active posts  

Home / Forums Index / WebmasterWorld / Website Analytics - Tracking and Logging
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved