Welcome to WebmasterWorld Guest from 22.214.171.124 , register , free tools , login , search , pro membership , help , library , announcements , recent posts , open posts Pubcon Platinum Sponsor 2014
script that meets these needs warrior01
Does anyone know of a search engine that lets you restrict the directories to seach and that is php and GPL or GNU (free for redistribute)?
Thanks in advance,
That one is a good start. PHPdig [ phpdig.toiletoine.net]
It's free (GNU/GPL) and it even seems (never used it myself, simply scanned the feature page) to follow the robots.txt file which should allow you to configure all the directory exclusion based on user-agent.
hope that helps