Forum Moderators: open

Message Too Old, No Replies

vang.net spider 1.6 (Spider 1.7/site@vang.net)

vang.net + soegning.dk - aggressive new spider

         

claus

10:27 am on Nov 3, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I've been visited a lot by this one lately:

User-Agent: vang.net spider 1.6 (Spider 1.7/site@vang.net)
Referrer: http*//www.soegning.dk/
IP: 62.58.32.94
Requests: GET + HTTP/1.0
-------------------
inetnum: 62.58.32.64 - 62.58.32.95
netname: VERSATEL-CUST-KOVOKS
descr: Versatel FBI customer
country: NL

Note: i've delinked the referrer link, it does have a colon in stead of an asterisk

The spider seems very aggressive - sofar this month it has generated more hits than UCMore, yesterday it retrieved more files than Googlebot (which is the most agressive one, albeit one i don't mind about). It's not possible to violate the robots.txt on the site at the moment, as all html is allowed, so i'm afraid i don't know if it behaves or not. It does take a lot of bandwidth though.

At the referrer address a new Danish Search Engine is located. It does seem to work. At the address "www.vang.net" a Dutch company is located, and their homepage is only in Dutch, so i don't understand any of it.

To me it seems like a real SE spider - and also a log file spammer.

Any thoughts?

/claus

troels nybo nielsen

11:44 am on Nov 3, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I haven't seen it read my robots.txt. I'm gonna ban it. My laziness is the only reason that I haven't done already.

soegning.dk looks like an ODP clone.