Welcome to WebmasterWorld Guest from

Forum Moderators: Ocean10000 & incrediBILL & keyplyr

Message Too Old, No Replies

Why block a Spider?



11:58 am on May 17, 2001 (gmt 0)

Inactive Member
Account Expired


Why would anyone block spiders, i thought they where good?
12:40 pm on May 17, 2001 (gmt 0)

Junior Member

10+ Year Member

joined:Aug 2, 2000
votes: 0

There are several reasons you would block a spider. Some spiders take up too much of your bandwidth. Sometimes you don't want spiders in some areas of you site. Your site may only be relevant for one country and you don't want the site listed if search engines of other countries.
12:50 pm on May 21, 2001 (gmt 0)

Junior Member

10+ Year Member

joined:May 16, 2001
votes: 0

Although this might be off topic... how do you go about creating a spider? Just curious..