Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

Using wild card in robots

Can you add * to ip?

5:28 pm on Apr 13, 2004 (gmt 0)

Preferred Member

10+ Year Member

joined:Dec 1, 2003
votes: 0

Can you do this to block all combinations of the last part of an ip?
5:37 pm on Apr 13, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
votes: 0


I'm afraid you have confused robots.txt with .htaccess and other methods of access control.
The objects to be disallowed in robots.txt are robot names and resources -- such as pages -- on your site.

User-agent: Scooter
Disallow: /igi-bin/
Disallow: /private/

Try a search on WebmasterWorld for "mod_rewrite user_agent" or ".htaccess Deny", or if you're on an IIS server, "ISAPI rewrite".