Page is a not externally linkable
bull - 12:32 pm on Sep 25, 2002 (gmt 0)
won't work always. had this one today, grabbed some hundred pages from my beloved site: p5084d1b1.dip.t-dialin.net - - [25/Sep/2002:13:34:40 +0200] "GET /_omitted.htm HTTP/1.0" 200 2373 www.mydomain.net "_omitted.htm" "Mozilla/4.5 (compatible; HTTrack 3.0x; Windows 98)" "-" So, this might be better as far as I can see:
RewriteCond %{HTTP_USER_AGENT} httrack [OR]
RewriteCond %{HTTP_USER_AGENT} .*httrack.* [NC,OR]
Besides, HTTrack seems to respect robots.txt