Forum Moderators: open

Message Too Old, No Replies

gamma.ucsd.edu tests multiple bad bots

Nmap Scripting Engine, favicon Internet survey research

         

Pfui

5:02 pm on Dec 24, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



From the University of California, San Diego.* Bots possibly related beyond sharing the same Host because hit patterns were identical (top level, then favicon, then gone):

gamma.ucsd.edu
Mozilla/5.0 (compatible; Nmap Scripting Engine; http://nmap.org/book/nse.html)
robots.txt? NO

gamma.ucsd.edu
Mozilla/5.0 (Linux; favicon Internet survey research; bmenrigh@ucsd.edu)
robots.txt? NO

*Prior bit on blocking troublesome bots from .edu subdomains here [webmasterworld.com].

Pfui

5:25 am on Jan 21, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



gamma.ucsd.edu
Mozilla/5.0 (compatible; favicon.ico Internet survey; research only)

robots.txt? NO

Hit root, then ~3 minutes later, hit favicon.ico.

FWIW: I get so tired of university profs/students/employees ignoring Web standards like the Robots Exclusion Protocol [robotstxt.org] and running roughshod all over the place. Try raising the bar!

See also, courtesy of a spelling-challenged bot-runner at stanford.edu: TwitterReseach [webmasterworld.com] [sic]

wilderness

5:46 am on Jan 21, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Pfui,
The "trend" has always been to portray and view universities as a higher learning culture. As webmasters we're
"supposed" to clearly allow exceptions?

However. . .if you think of these same universities as 3rd party groups funded by research grants?
I can assure (from some ten years worth of experience) that you'll view their visits in your logs quite differently.

keyplyr

5:53 am on Jan 21, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Apologies for my former colleagues at UCSD. Academics are well known to be sanctimonious, holding themselves unaccountable for acts done in the interest of research and education. Doubtful this will ever change.