homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

He.net, anybody know who he is.

 3:28 pm on Apr 26, 2001 (gmt 0)


He.net or analysis.he.net has been to several of my sites doing deep crawls. Any one know who it is?




 4:37 pm on Apr 26, 2001 (gmt 0)

Been seen before, no definitive answers:



 5:05 pm on Apr 26, 2001 (gmt 0)

They've been making the rounds laity.
This is one of the post that talks about it. I think there are two others. If you plug in their IP it will throw you to a splash page that talks about the server farm in Fremont. I know they are crawling from two separate IPs. At first I thought they might be doing some clever marketing by causing a buz with their crawling. But I really don't know what they are up to yet.


 8:20 pm on Apr 28, 2001 (gmt 0)

Its hammering one of my sites, doing faulty requests for fragments of javascript and generally annoying me :(
Strange thing is it has a UA of Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0).
I am very tempted to ban them.


 9:39 pm on Apr 29, 2001 (gmt 0)

Any ideas if it's worth blocking this IP...is there a good way of banning them *as if there is a good way of banning, but hey* Thanks for all the help ;)


 9:48 pm on Apr 29, 2001 (gmt 0)

If you are using a *nix system you can use the .htaccess file to ban them and it will work unless they start coming from a different IP.


 11:13 pm on Apr 29, 2001 (gmt 0)

I contacted the huuricane electric abuse department. After they deep crawled my site super fast 3 times. Here was there response: Thank you for your feedback/interest. analysis.he.net is an experimental
web indexing tool. We take a number of precautions to ensure that the
load we cause your web server is minimized such as only accessing one URL
at your domain name at a time. We appologize for any confusion we may
have caused.
And here is the email address of the guy who responded: Scott Nelson <scottn@he.net>


 1:33 am on Apr 30, 2001 (gmt 0)

The email was a big [url=www.he.net/~scottn/]help[/url]. Finally something to work with...

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved