Forum Moderators: open

Message Too Old, No Replies

Bot with blank referrer and UA - Microsoft IP

It's back..

         

LunaC

6:02 pm on Jan 29, 2007 (gmt 0)

10+ Year Member



I saw this ages ago and it seemed to stop, but the last while it's started again. So far I haven't seen it request robots.txt and it's getting blocked by my "no referrer and UA.. go away" policy, but I'm starting to wonder if that really is the best idea.

Yesterday it requested hundreds of files, never images, scripts or css, just pages and it seems to grab these pages in short bursts.

Does anyone know what it is, and any opinions whether I should allow it access?

Here's a sample:
131.107.x.xx - - [28/Jan/2007:05:41:41 -0800] "GET /file.html HTTP/1.1" 403 646 "-" "-"

Mokita

12:52 am on Jan 30, 2007 (gmt 0)