Forum Moderators: DixonJones
RewriteCond %{HTTP_USER_AGENT} ^Microsoft.(Data¦URL) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Microsoft-WebDAV-MiniRedir [NC,OR]
RewriteCond %{HTTP_USER_AGENT} MS.*FrontPage [NC]
rewriterule !^robots\.txt$ - [F]
<added>
Don't forget to replace the broken pipe(¦) with the one from your keyboard.
</added>
That line translates, "If the requested resource is NOT robots.txt, leave the URL alone, but return a 403-Forbidden server response."
If you use a custom 403 page, then it should also be excluded as with robots.txt:
RewriteRule !^(robots\.txt¦custom-403\.html)$ - [F]
Jim