| Welcome to WebmasterWorld Guest from 22.214.171.124 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Become a Pro Member|
|Using wild card in robots|
Can you add * to ip?
Can you do this to block all combinations of the last part of an ip?
I'm afraid you have confused robots.txt with .htaccess and other methods of access control.
The objects to be disallowed in robots.txt are robot names and resources -- such as pages -- on your site.
Try a search on WebmasterWorld for "mod_rewrite user_agent" or ".htaccess Deny", or if you're on an IIS server, "ISAPI rewrite".
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved