Forum Moderators: open

Message Too Old, No Replies

fembot

Is this new?

         

Mokita

4:13 am on Mar 12, 2007 (gmt 0)

10+ Year Member



Full UA: fembot (myd@cs.stanford.edu)

Requested robots.txt plus one (allowed) file. IP is 207.7.153.nn, belonging to "United Layer", which looks like they are a server farm/colo.

I wasn't able to find anything about it in Google - anyone have any further info?

zCat

6:57 pm on Mar 12, 2007 (gmt 0)

10+ Year Member



No, but it does seem to have some connection with "bender/Nutch-0.8.1 (myd@cs.stanford.edu)" (see my post [webmasterworld.com...] ).

Mokita

11:01 pm on Mar 12, 2007 (gmt 0)

10+ Year Member



Thanks zCat - yes "bender" came by the same site a couple of hours after I made my post.

I added both bender and fembot to robots.txt in other sites that it had not yet visited and bender visited one of them last night, asked for robots.txt but no files.

So it might be an obedient bot, but as I don't have any idea its purpose is, I'll probably ban the CIDR for the colo.