homepage Welcome to WebmasterWorld Guest from 50.19.172.0
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Spiders
He.net, anybody know who he is.
woodrow

10+ Year Member



 
Msg#: 595 posted 3:28 pm on Apr 26, 2001 (gmt 0)

Hello,

He.net or analysis.he.net has been to several of my sites doing deep crawls. Any one know who it is?

Thanks

 

bobriggs

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 595 posted 4:37 pm on Apr 26, 2001 (gmt 0)

Been seen before, no definitive answers:

[webmasterworld.com...]

littleman

WebmasterWorld Senior Member littleman us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 595 posted 5:05 pm on Apr 26, 2001 (gmt 0)

They've been making the rounds laity.
[webmasterworld.com...]
This is one of the post that talks about it. I think there are two others. If you plug in their IP it will throw you to a splash page that talks about the server farm in Fremont. I know they are crawling from two separate IPs. At first I thought they might be doing some clever marketing by causing a buz with their crawling. But I really don't know what they are up to yet.

theperlyking

10+ Year Member



 
Msg#: 595 posted 8:20 pm on Apr 28, 2001 (gmt 0)

Its hammering one of my sites, doing faulty requests for fragments of javascript and generally annoying me :(
Strange thing is it has a UA of Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0).
I am very tempted to ban them.

digitalgirl

10+ Year Member



 
Msg#: 595 posted 9:39 pm on Apr 29, 2001 (gmt 0)

Any ideas if it's worth blocking this IP...is there a good way of banning them *as if there is a good way of banning, but hey* Thanks for all the help ;)

theperlyking

10+ Year Member



 
Msg#: 595 posted 9:48 pm on Apr 29, 2001 (gmt 0)

If you are using a *nix system you can use the .htaccess file to ban them and it will work unless they start coming from a different IP.

bot_watcher

10+ Year Member



 
Msg#: 595 posted 11:13 pm on Apr 29, 2001 (gmt 0)

I contacted the huuricane electric abuse department. After they deep crawled my site super fast 3 times. Here was there response: Thank you for your feedback/interest. analysis.he.net is an experimental
web indexing tool. We take a number of precautions to ensure that the
load we cause your web server is minimized such as only accessing one URL
at your domain name at a time. We appologize for any confusion we may
have caused.
And here is the email address of the guy who responded: Scott Nelson <scottn@he.net>

Froggyman



 
Msg#: 595 posted 1:33 am on Apr 30, 2001 (gmt 0)

The email was a big [url=www.he.net/~scottn/]help[/url]. Finally something to work with...

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved