Forum Moderators: open

Message Too Old, No Replies

Spoofed Mozilla/4.72

Acts like a robot, doesn't send a Host header

         

Kimihia

1:46 am on Feb 14, 2001 (gmt 0)



I've had the user agent "Mozilla/4.72 [en] (X11; U; Linux 2.2.12-20 i686)" attempting to spider my server.

It doesn't send a Host header (which causes trouble attempting to index my server which hosts 20+ domains).

It came from the addresses 206.191.170.233, 206.64.105.81, 206.229.153.81, 208.51.235.81, and 206.98.113.81.

Never looked at robots.txt

Any thoughts?

Air

3:09 am on Feb 14, 2001 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hi kimihia,

You'll find some useful info on that spider HERE [webmasterworld.com]. It was reported by some other members of WebmasterWorld in that thread, and there is a reply from the company running the spider included as well.