Forum Moderators: phranque
I set up my site so that anyone trying to 'offline' bookmark certain pages would get kicked to the homepage. Bad idea, I killed my CTR on all of those ads now! Good thing they are random, but still!
So I am going to set up a warning page instead, but I'm wondering if I'm catching all of these people now. Does anyone have a list of User_Agent keys that identify that someone is trying to bookmark the page for offline viewing?
There are different methods of blocking "site downloader" programs from accessing your site.
You shouldn't interfere with someone who just wants to bookmark that particular page in their browser as it is annoying.
Thanks.
What I'm seeing is the bookmark for 'offline viewing' feature (MSIEcrawler) and I also see some other strange UAs but I'm not sure of all of the downloading programs.
I may have to set up a check to see if someone is hitting a page more than 3 times per second for example, and then feed them a warning page.
That would probably solve the entire problem, but I prefer the UA check.
Is there a list of UAs somewhere online?
Advanced Site Crawler
Black Widow
Custo
dloader
download b
Download Demon
Download Ninja
Downloader
Dual Proxy
EasyDL
eCatch
EirGrabber
Express WebPictures
ExtractorPro
Ez Auto Downloader
FlashGet
GetRight
HLoader
HMView
HtmlDownload
HTTrack
Image Stripper
Image Sucker
Indy Library
InterGET
Internet Ninja
ISSB Website Loader
LeechFTP
Mass Downloader
Microsoft URL Control
Microsoft Data Access
MSProxy
oBot
Offline Explorer
Offline Navigator
PageGrabber
PICgrabber
Program Shareware
Rafa Bot
RealDownload
Reaper
Schmozilla
SecretBrowser
SiteSnagger
SiteSucker
SmartDownload
StreamRipper32
Stripper
Sucker
Teleport Pro
WebCopier
Web Downloader
Web Image Collector
Web Retriever
Web Sucker
Webcature
Webdup
webloader
WebPix
WebSauger
Website Capture
Website eXtractor
Website Quester
WebReaper
WebStripper
<added>
Here's another list:
http*//download.com.com/3120-20-0.html?qt=website+download&tg=dl-2001&search=+Go!+
</added>
Thanks
