Forum Moderators: open

Message Too Old, No Replies

COAST WebMaster

         

Pfui

8:31 pm on Nov 9, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



protect-129-174-52-nn.mesa.gmu.edu
COAST WebMaster/4.1.1.1 (Windows NT)

robots.txt? NO

A link-checking Swiss Army Knife'esque thing. Too bad for them there's only room for one webmaster on my sites. Ahem.

GaryK

1:23 am on Nov 10, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Sure wish I could blindly ban everything from a edu TLD. Unfortunately I have one site that gets a lot of human-powered visits from edu (and gov) TLDs so I have to be sure I'm only banning bots not humans.

Pfui

3:22 am on Nov 10, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



The vast majority of my troublesome .edu hits come from computer science types running all sorts of bots on university servers, including pesky 'personal' Nutch projects. Ergo, it's redirection detention for all of 'em:

RewriteCond %{REMOTE_HOST} \.cs\. [OR]
RewriteCond %{REMOTE_HOST} \.cse\. [OR]

(Note no NC.) If you'd prefer school-specific snippets:

RewriteCond %{REMOTE_HOST} \.cs\.univname\.edu$ [OR]
RewriteCond %{REMOTE_HOST} \.cse\.univname\.edu$ [OR]

Whitelist as need be.