Welcome to WebmasterWorld Guest from

Forum Moderators: Ocean10000 & incrediBILL & phranque

Message Too Old, No Replies

How to use a .htaccess file

to keep spiders out

4:07 pm on May 7, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Jan 13, 2003
votes: 0

I have heard that I can use a .htaccess file to keep all but ONE search engine spiders out of my domain. I am going to use a robots.txt to allow only the ONE robot i want to visit but somebody told me that they would use a .htaccess file just in case.

Could anybody please give me any pointer on how to create one. I did look through some of the old threads but couldn't see anything.

3:39 am on May 8, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 21, 1999
votes: 0

A few tweaks to the Close to perfect .htaccess ban list [webmasterworld.com] should meet your needs.