homepage Welcome to WebmasterWorld Guest from 50.19.172.0
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Surfairy
another one for you DNS junkies
vis

10+ Year Member



 
Msg#: 455 posted 11:36 am on Mar 15, 2001 (gmt 0)

My normal trails run dry on this one:

212.150.52.26 HTTP/1.1 Surfairy

Just grabbed index page.

I get as far as it being an Israel domain, then nothing.

A combination of household detergents? (probably only relevant to UK :))

 

Air

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 455 posted 4:56 pm on Mar 15, 2001 (gmt 0)

Looks like they are an ISP:

[barak-online.net...]

Froggyman



 
Msg#: 455 posted 1:51 am on Mar 16, 2001 (gmt 0)

Funny, I was going to mention this spider as it graced my index page for the first time today.

[divago.com...]

vis

10+ Year Member



 
Msg#: 455 posted 10:07 am on Mar 16, 2001 (gmt 0)

OK, the fairy can stay- since it might generate some traffic every other blue moon.

Not sure about this one though:

212.209.107.180
InternetAmi+IOR/0.2+(http://internetami.se/ior.html)

Since my website is pretty badly ritten I think NOT banning it would hamper their efforts in obtaining accurate translations.

SugarBane

10+ Year Member



 
Msg#: 455 posted 2:23 pm on Apr 3, 2001 (gmt 0)

Regarding Surfairy..

I did some looking into this one when it hit my site a few weeks back. It turns out that it's a plugin/3rd party application that some ISP's can purchase and run for their clients whereby if someone uses this search mode, it will come up with suggested websites that closely match the string entered if the URL the client types in leads to a 404 return.

Surfairy keeps it's own database of websites which it spiders regularly to provide an updated database to ISP's using this feature. Kinda interesting. Seems to not be agressive so I don't see any reason to ban this one.

For more information see: [divago.com...]

Follow the Flash presentation for ISP's. It explains all about Surfairy.

SB.

effip

10+ Year Member



 
Msg#: 455 posted 12:21 pm on Apr 18, 2001 (gmt 0)

An official explanation from Divago Inc.
========================================

As you already noted, Surfairy is an addon to the Internet Explorer which will replace the error messages (40x, 50x and DNS errors) with an inteligent answer. The system can also operate on ISP sites without a client.
In order to provide the inteligent answer we maintain a database of active URLs. We don't scan the site and we only check for home page existance so the spider is minimally intrusive.
The client is available on our site for download so you are invited to try it.
If you need more information feel free to send me an email to effi@divago.com

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved