Page is a not externally linkable
Drreggae - 11:20 am on Jan 4, 2008 (gmt 0)
regards! Btw, gonna ask this in another subforum, but.. is that crawler: 78.137.163.133 coming from digiweb.ie with by any chance a crawler from ask.com? I wonder, cause some This 78.137.163.133 is all over the web in visible access logs [edited by: Drreggae at 11:27 am (utc) on Jan. 4, 2008]
Thanks Vivek!
I just came back here to post that indeed that crawler now
started to act normal again since, a small week or so.
Now we hope to see the same traffic as before back again;-)
(sure part is the holidays and end of Q4..:-()
"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a1) Gecko/20070308 Minefield/3.0a1"
ask crawlers indeed come from .ie if I am correct, and always
use Minefield..
etc, ( ip-78-137-163-133.dedi.digiweb.ie ).
Since it does no identify itself, I now block it via .htaccess
but surely would be not doing that if I knew if was from
ask.com..