Forum Moderators: open

Message Too Old, No Replies

How to kill the Mattie MP3 Spider

This Spider is eating my bandwidth

         

hellier

11:28 am on Jan 23, 2002 (gmt 0)

10+ Year Member



Can anyone advise me on how to stop a search engine spider called 'Mattie'. it's sole purpose is to sniff out MP3's.

I have tried to visit the guy's web site and email. (Both are dead).

This spider eats up considerable bandwidth on my server. Any help would be greatly appreciated.

M

agerhart

2:22 pm on Jan 23, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Welcome to WmW Hellier.

Have you disallowed to bot in your robots.txt?

gethan

3:43 pm on Jan 23, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hi Hellier,

[webmasterworld.com...]

Sounds like you have another spider to add to toolmans ban list. The basic advantage of using the method in the post above is that you control which spiders are allowed access - whereas with robots.txt your reliant on the spider to follow your instructions.

Caveats: needs apache with mod_rewrite, slight performance hit.