Welcome to WebmasterWorld Guest from 220.127.116.11 , register , free tools , login , search , pro membership , help , library , announcements , recent posts , open posts Become a Pro Member
New FAST spider? hmmmm? roscoepico msg:225109 2:17 pm on Jun 12, 2001 (gmt 0) Im seeing the following in my logs
Visitor IP: 18.104.22.168
UA: FAST-WebCrawler/2.2.6 (email@example.com; [ ...] ah-ha.com
Does anyone what what type of relationship FAST and AH-HA have? Should I be feeding them optimized pages for FAST?
Any help is appreciated
Steph msg:225110 2:59 pm on Jun 12, 2001 (gmt 0)
i saw the same, and i saw that in there FAQ
"The FAST Web Crawler is a distributed crawler that collects content from the web for use by ah-ha.com's Web Search."
littleman msg:225111 4:43 pm on Jun 12, 2001 (gmt 0)
This has been an issue for me for over a year now. The databases are not at all in sink. It looks to me like they are independent of each other, but there is a possibility that Fast will do a uni-directional data dump -- or even some cross referencing.