| Welcome to WebmasterWorld Guest from 126.96.36.199 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Pubcon Platinum Sponsor 2014|
|Mozilla/5.0 (compatible; ScoutJet; http://www.scout jet.com/)|
Letting it in, or blocking it?
This was sniffing round several sites I work on in the last few days.
ScoutJet web crawler. ScoutJet is the web crawler for a new Silicon Valley based search engine company created by the founders of DMOZ and Topix.
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved