Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

Using wild card in robots

Can you add * to ip?



5:28 pm on Apr 13, 2004 (gmt 0)

10+ Year Member

Can you do this to block all combinations of the last part of an ip?


5:37 pm on Apr 13, 2004 (gmt 0)

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member


I'm afraid you have confused robots.txt with .htaccess and other methods of access control.
The objects to be disallowed in robots.txt are robot names and resources -- such as pages -- on your site.

User-agent: Scooter
Disallow: /igi-bin/
Disallow: /private/

Try a search on WebmasterWorld for "mod_rewrite user_agent" or ".htaccess Deny", or if you're on an IIS server, "ISAPI rewrite".



Featured Threads

Hot Threads This Week

Hot Threads This Month