Forum Moderators: open

Message Too Old, No Replies

hl_ftien_spider

Bot from China

         

lammert

1:13 am on Feb 26, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



This bot came from IP 211.100.29.23. Reads robots.txt but doesn't obey its contents. Read 413 pages from one of my sites (almost 100% of the content) in 3 minutes.

Hetta

6:03 pm on Mar 9, 2006 (gmt 0)

10+ Year Member



This bot hit my site today, picked up robots.txt, then picked up a few dozen pages - and tried to pick up about 1800 more pages after it hit a bot trap.

All of which it did within 4 minutes.

The IP is the same as you had, and it resolves to net263.com in China.

I've banned it.

SpliFF

1:42 pm on Mar 12, 2006 (gmt 0)

10+ Year Member



This bot is still around and it appears to be slower but I've banned it for creating spurious errors (by trying to index javascript: links). I banned Omni explorer for the same reason.