Forum Moderators: phranque

Message Too Old, No Replies

Banning Site Rippers Using .htaccess

Can the .htaccess file be modified to ban rippers

         

gijimbo

7:44 pm on Jul 16, 2005 (gmt 0)

10+ Year Member



I would like to know if there is anyway the .htaccess file can be configured to prevent site ripping programs from being used against your site.

Last weekend I just had a major university from England rip my site. I have the universities entire IP range blocked. But that only works if the person is at the university.

Any thoughts would be greatly appreciated. I don't not want to encrypt the entire site.

Thanks,
Jim

jdMorgan

3:21 pm on Jul 18, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



By itself, .htaccess is not capable of adaptive behaviour. It is a static file. Plus, HTTP is a stateless protocol; That is, each HTTP request for an HTML page or an image exists on its own; The server handles each one separately without any knowledge of previous or subsequent requests.

What this means is that by itself, .htaccess cannot "detect" activity that spans more than one HTTP request, such as "site ripping."

However, there are script-based techniques that can detect ripping and modify your .htaccess file to stop the current exploit and prevent further exploits from the same IP address. Two of these scripts and the site modifications needed to support them are described in threads here at WebmasterWorld: See message #16 of this thread [webmasterworld.com] to get started.

[Added]
One of the scripts mentioned in that thread has been updated. See [webmasterworld.com...]

For a simple user-agent-based approach, see our classic "Close to perfect .htaccess ban list" thread (in four parts): [webmasterworld.com...]
[/added]

Jim