Welcome to WebmasterWorld Guest from 54.81.166.196
Forum Moderators: Ocean10000 & incrediBILL & phranque
For example if the url was the index page i was thinking something like
RewriteCond %{THE_REQUEST} !^http://www.domain.com/ [OR]
RewriteCond %{HTTP_USER_AGENT} ^-?$ [OR]
RewriteCond %{REMOTE_ADDR} ^x.x.x.x/x
RewriteRule .* - [F]
but i really struggle with this stuff.. :-(
# If user-agent is blank or "-"
RewriteCond %{HTTP_USER_AGENT} ^-?$
# and if remote address is 123.45.67.0 through 123.45.67.127
RewriteCond %{REMOTE_ADDR} ^123\.45\.67\.([1-9]?[0-9]¦1[01][0-9]¦12[0-7])$
RewriteRule ^always_the_same_page\.html$ - [F]
Replace the broken pipe "¦" characters in that pattern with solid pipes before use; Posting on this forum modifies the pipe characters.
Put a pattern matching the URL-path for the initially-requested page in the RewriteRule as shown.
You could also change the Rule so that it directly calls your bad-bot script instead of issuing a single 403-Forbidden response, for example:
RewriteRule ^always_the_same_page\.html$ /path_to_bad-bot.pl [L]
If the bad-bot script isn't being triggered soon enough, then consider whether your embedded 'trap' URLs are numerous enough, well-placed high in the source code and throughout, vary enough in URL appearance, and "look interesting enough" to a harvester. There are quite a few ways to embed invisible trap links, and you should use several, with all Disallowed in robots.txt, and all rewritten to your bad-bot script.
Jim