Forum Moderators: open

Message Too Old, No Replies

How do I set up a spider trap?

badwidth theft

         

Digger

11:36 pm on May 21, 2003 (gmt 0)

10+ Year Member



Could someone tell me how to set up an automated spider trap with .htaccess?

I'm on an apache server. I use cpanel and see unknown spiders going throug my pages.

SEO practioner

11:50 pm on May 21, 2003 (gmt 0)

10+ Year Member



Hello Digger and welcome to webmaster world!

I think this will help you:

[webmasterworld.com...] - size 46kb 2003

Good luck

jatar_k

11:55 pm on May 21, 2003 (gmt 0)

WebmasterWorld Administrator 10+ Year Member



how about this one
modified "bad-bot" script blocks site downloads [webmasterworld.com]

I posted this particular one becaus eit has links to all the others I was looking for.