Forum Moderators: open
I've had some problems with what I thought was a scraper from Hurricane Electric before, and I've never noticed the Gigabot user agent. Is this legit? I'm all for helping out Gigablast and all, but I want to be sure Gigablast is really behind this.
I asked Matt if there's any connection between Hurricane Electric and [Gigablast] other than they seem to be using [Gigablast's] software.
I can't quote the reply I got but he gave me an obviously obfuscated reply about, having some servers hosted there. I have no idea which servers he's referring to or who "we" refers to.
[Edited for clarity.]
I run IIS so .htaccess is of no help to me. I do use an httpd.ini file from ISAPI_Rewrite to essentially do what .htaccess does.
But I'm not looking for information on how to block anything. I can handle blocking bots just fine.
All I want to know is if I understand the relationship between Hurricane Electric and all these badly behaved bots correctly so that I can provide the people who download my browscap.ini file each week with accurate information.
:)
I would think it could be any one of tens-of-thousands of Webmasters with an account there. To ban the entire netrange would seem (IMHO) overkill.
Friday
run their own spider(s)?
It's really irrelavant.
They offer hosting and fail to enforce the UAG that their hosting customers agree to.
" 2. Any use which interferes with the server's ability to function in its primary purpose of publishing web documents is prohibited."
"5. Use of XXXXXXXX XXXXXXX's facilities to commit network abuse (including, but not limited to, denial of service attacks such as ping bombing, email bombing, "smurf", "winnuke", "land", "teardrop", etc.) or otherwise compromise the security of hosts or networks is prohibited."
I may be mistaken, however when I present my websites and their content, my intent is to deliver to genuine IP ranges.
The other other "web sites" that I personally desire to visit my pages, are those that identify themselves and run parallel to the goals for my websites.
Any other website on non-parallel visitor can eat 403's.
Just to be clear, this wouldn't prevent any of those "tens-of-thousands of webmasters" from visiting his site -- when they're online they're not connecting through servers at HE. It'd block any bots operating from those servers; it wouldn't have any effect on regular human users.
As a long-time customer of theirs, I can assure you that theyu DO enfore their policies. You get ONEW warning and then you're history. I've had numerous complaints of clients (I used to be in the Web Dev field) get me in trouble and the folks at He.Net do not take complaints lightly
I can only speak as far back as the last last time I got a warning from them regarding a client site (about six months ago - and that was because a kracker had exploited a contact-form script to email spam from his account.
BUT, someone has to complain first.
This is just my own personal experience with their abuse department.
Your mileage may vary.
If all HE does is host websites then I've lost nothing and gained CPU cycles by banning all of HE's IP Addresses, thus keeping out all the badly behaved bots that seem to emanate from HE IP Addresses.
Words in this case are meaningless. Experience has taught me that not only doesn't HE abide by its own rules, it's also complicit in the operation of a lot of these bots. Too many of the domain names I trace these bots back to are registered to the same person that appears to own HE.