Forum Moderators: open

Message Too Old, No Replies

Gigabot/2.0

Is this legit?

         

dataguy

11:32 pm on Jul 17, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I've got some bots going wild on one of my sites from about 20 different IP's, all registered to Hurricane Electric and with a UA of 'Gigabot/2.0'.

I've had some problems with what I thought was a scraper from Hurricane Electric before, and I've never noticed the Gigabot user agent. Is this legit? I'm all for helping out Gigablast and all, but I want to be sure Gigablast is really behind this.

GaryK

12:51 pm on Jul 18, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Gigabot is owned by Matt Wells and was founded in 2000. His software is for sale so perhaps Hurricane Electric purchased it. Will you please share the IP Addresses you saw? Thanks.

EDIT: I just e-mailed Giga about this situation. I'll provide a summary of their reply, if I get one.

dataguy

1:07 pm on Jul 18, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



IP's ranged from 64.62.168.15 to 64.62.168.70.

GaryK

1:13 pm on Jul 18, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Sad to say those IPs are registered to Hurricane Electric with a reference to Matt Wells so I'd have to say there is a connection between them. Time for me to ban Giga's IP Addresses too. Thanks for the info.

GaryK

4:36 pm on Jul 18, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I just received a reply from Matt. He completely ignored my question about there being a link between Gigablast and Hurricane Electric. I've asked him to please answer my question. I won't be rude to him so that's really all I can do.

wilderness

4:48 pm on Jul 18, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



There's and old (not too) four page thread on Hurricane.
Best thing any webmaster may do is:

RewriteCond %{REMOTE_ADDR} ^64\.62\.(12[8-9]¦1[3-9][0-9]¦2[0-5][0-9])\. [OR]

GaryK

5:10 pm on Jul 18, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Yup, I know all about Hurricane Electric (HE). I'm trying to confirm a connection between HE and an apparently legit search engine called Gigablast. :)

I asked Matt if there's any connection between Hurricane Electric and [Gigablast] other than they seem to be using [Gigablast's] software.

I can't quote the reply I got but he gave me an obviously obfuscated reply about, having some servers hosted there. I have no idea which servers he's referring to or who "we" refers to.

[Edited for clarity.]

wilderness

5:24 pm on Jul 18, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



"servers hosted?"

Hurricane Electric HURRICANE-4 (NET-64-62-128-0-1)
64.62.128.0 - 64.62.255.255
BigBiz Internet Services HURRICANE-CE0004-2A9 (NET-64-62-128-0-2)
64.62.128.0 - 64.62.128.255

Just do a google on the second line

GaryK

5:44 pm on Jul 18, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



It's Monday and I'm dense. Please spoon-feed me, LOL.

I see where Google shows lots of different website hosts apparently using IP Addresses that are registered to HE. Does this mean HE is the "parent webhost" and these other companies are the evil spawn of HE?

GaryK

9:31 pm on Jul 19, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Can someone please confirm my understanding about HE as posted above? Thanks.

GaryK

12:02 am on Jul 21, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I guess no one can help me with this issue. Oh well.

kevinpate

1:56 am on Jul 21, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>no one can help me

The suggested htaccess code addition in msg 6 would appear to fal within the envelope of giving aid.

keyplyr

7:12 am on Jul 21, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Hurricane also sends OmniExplorer_Bot from: 65.19.128.0 - 65.19.191.255

RewriteCond %{REMOTE_ADDR} ^65\.19\.1[2-9][0-9]\. [OR]
This work?

GaryK

2:02 pm on Jul 21, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Thank you for replying Kevin.

I run IIS so .htaccess is of no help to me. I do use an httpd.ini file from ISAPI_Rewrite to essentially do what .htaccess does.

But I'm not looking for information on how to block anything. I can handle blocking bots just fine.

All I want to know is if I understand the relationship between Hurricane Electric and all these badly behaved bots correctly so that I can provide the people who download my browscap.ini file each week with accurate information.

:)

Friday

10:25 pm on Aug 3, 2005 (gmt 0)

10+ Year Member



Huricane Electric is a HUGE website hostiong company (In fact, I recommend them for anyone looking for a hosting co. - I've used them for years for many sites).

I would think it could be any one of tens-of-thousands of Webmasters with an account there. To ban the entire netrange would seem (IMHO) overkill.
Friday

wilderness

10:41 pm on Aug 3, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



To ban the entire netrange would seem (IMHO) overkill.

Than I guess denying RIPE and APNIC is out of the question ;)

Friday

11:21 pm on Aug 3, 2005 (gmt 0)

10+ Year Member



Wilderness,

Do I detect a hint of sarcasm?
;-)

Seroiusly, Does He.Net (the hosting co., itself) run their own spider(s)? Or is it just an account holder?

Friday

wilderness

11:31 pm on Aug 3, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



run their own spider(s)?

It's really irrelavant.
They offer hosting and fail to enforce the UAG that their hosting customers agree to.

" 2. Any use which interferes with the server's ability to function in its primary purpose of publishing web documents is prohibited."

"5. Use of XXXXXXXX XXXXXXX's facilities to commit network abuse (including, but not limited to, denial of service attacks such as ping bombing, email bombing, "smurf", "winnuke", "land", "teardrop", etc.) or otherwise compromise the security of hosts or networks is prohibited."

I may be mistaken, however when I present my websites and their content, my intent is to deliver to genuine IP ranges.

The other other "web sites" that I personally desire to visit my pages, are those that identify themselves and run parallel to the goals for my websites.

Any other website on non-parallel visitor can eat 403's.

JayC

11:42 pm on Aug 3, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>> I would think it could be any one of tens-of-thousands of Webmasters with an account there. To ban the entire netrange would seem (IMHO) overkill.

Just to be clear, this wouldn't prevent any of those "tens-of-thousands of webmasters" from visiting his site -- when they're online they're not connecting through servers at HE. It'd block any bots operating from those servers; it wouldn't have any effect on regular human users.

Friday

1:02 am on Aug 4, 2005 (gmt 0)

10+ Year Member



Wilderness,

As a long-time customer of theirs, I can assure you that theyu DO enfore their policies. You get ONEW warning and then you're history. I've had numerous complaints of clients (I used to be in the Web Dev field) get me in trouble and the folks at He.Net do not take complaints lightly

I can only speak as far back as the last last time I got a warning from them regarding a client site (about six months ago - and that was because a kracker had exploited a contact-form script to email spam from his account.

BUT, someone has to complain first.

This is just my own personal experience with their abuse department.
Your mileage may vary.

Friday

1:14 am on Aug 4, 2005 (gmt 0)

10+ Year Member



JayC
Good point. I don't know what I was thinking. Long day, I guess.
;-)
Cheers,
Friday

GaryK

1:48 am on Aug 4, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



The way I see it if I want to ban all of HE's IP Addresses because I've had nothing but bad experiences with them, I can and will do so.

If all HE does is host websites then I've lost nothing and gained CPU cycles by banning all of HE's IP Addresses, thus keeping out all the badly behaved bots that seem to emanate from HE IP Addresses.

Words in this case are meaningless. Experience has taught me that not only doesn't HE abide by its own rules, it's also complicit in the operation of a lot of these bots. Too many of the domain names I trace these bots back to are registered to the same person that appears to own HE.

Friday

2:16 am on Aug 4, 2005 (gmt 0)

10+ Year Member



JayC

Thanks for bringing that to my attention.,
I didn't realize that.
What is his "game/scam" (He.net's owner)?