homepage Welcome to WebmasterWorld Guest from 50.17.7.84
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Mozilla/5.0 (compatible; ScoutJet; http://www.scout jet.com/)
Letting it in, or blocking it?
g1smd




msg:3674865
 6:44 pm on Jun 14, 2008 (gmt 0)

This was sniffing round several sites I work on in the last few days.

ScoutJet web crawler. ScoutJet is the web crawler for a new Silicon Valley based search engine company created by the founders of DMOZ and Topix.

Opinions?

 

bumpaw




msg:3675542
 12:57 am on Jun 16, 2008 (gmt 0)

It surfaced on one of my sites day. Seems legit so I'll let it run a while.

wilderness




msg:3675546
 12:59 am on Jun 16, 2008 (gmt 0)

[webmasterworld.com...]

g1smd




msg:3676177
 6:54 pm on Jun 16, 2008 (gmt 0)

So I now see... :-)

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved