Forum Moderators: open

Message Too Old, No Replies

CamontSpider/1.0

         

GaryK

3:13 pm on Nov 16, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



CamontSpider/1.0 [epweb2.ph.bham.ac.uk...]
130.246.45.nnn
heplnc187.pp.rl.ac.uk
-----
netname: RAL-IP
descr: Science and Technology Facilites Council
descr: Rutherford Appleton Laboratory
descr: Atlas Centre
descr: Chilton, Didcot
descr: Oxfordshire, OX11 0QX
country: GB
-----
READ ROBOTS.TXT? Yes
OBEYED ROBOTS.TXT? Unknown
-----
YAAP: yet another academic project
Took default root page from multiple sites then left.

keyplyr

7:37 pm on Nov 16, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



From earlier thread: [webmasterworld.com...]

194.36.10.*
fe07.esc.qmul.ac.uk
CamontSpider/1.0 +http://epweb2.ph.bham.ac.uk/user/slater/camont/info.html
robots.txt: no

GaryK

12:09 am on Nov 17, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Oops I missed that. Seems it's reading robots.txt now though.