Forum Moderators: open

Message Too Old, No Replies

Unknown spiders visiting

         

Dave_A

9:47 pm on Aug 16, 2005 (gmt 0)

10+ Year Member



Hi I run a search engine in New Zealand and I have seen a heap of "New Spiders" turning up all over the place.
It would appear that a few people have started to play with a webspider that is avialable to download from Sourceforge where it is being developed as a real web spider.

The problem is that it appears that a heap of people have started to play with this software, changing it's User Agent signature and seeing what they can poke with it because it has the ability to remove it checking the robots.txt function from the script.
When thay start to spider websites with my user agent details I get really pissed off..
I have IP of three such users and it's just a question of what to do next?

volatilegx

1:23 pm on Aug 17, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



In my opinion (but I'm not an attorney), there's really only one recourse for you: register a trademark for your spider's user agent (be sure to indicate that the user agent is trademarked in the user agent string for maximum protection), then pursue violators with legal action.

It's either the above, or grin and bear it.

elcapitan

10:49 pm on Aug 28, 2005 (gmt 0)

10+ Year Member



no - that is imposibile. There is nothing you can do about that. Is like trying to ban porn from the internet.