Forum Moderators: bakedjake
Here is quote from a log file:
63.121.41.177 - - [05/Aug/2003:02:34:13 -0400] "HEAD / HTTP/1.0" 200 0 "-" "GalaxyBot/1.0 (http://www.galaxy.com/galaxybot.html)" "www.example.com"
63.121.41.176 - - [05/Aug/2003:02:34:15 -0400] "HEAD / HTTP/1.0" 200 0 "-" "GalaxyBot/1.0 (http://www.galaxy.com/galaxybot.html)" "www.example.com"
63.121.41.178 - - [05/Aug/2003:02:53:32 -0400] "HEAD / HTTP/1.0" 200 0 "-" "GalaxyBot/1.0 (http://www.galaxy.com/galaxybot.html)" "www.example.com"
63.121.41.178 - - [05/Aug/2003:02:53:32 -0400] "HEAD / HTTP/1.0" 200 0 "-" "GalaxyBot/1.0 (http://www.galaxy.com/galaxybot.html)" "www.example.com"
(those were actually 4 different domains hosted on the same IP/log file...
I just noticed, it also comes with the following UA:
63.121.41.176 - - [04/Aug/2003:03:06:08 -0400] "GET /robots.txt HTTP/1.0" 200 33 "-" "Mozilla/4.0 (compatible; MSIE 5.0; www.galaxy.com; www.example.com; +http://www.galaxy.com/info/crawler.html)" "www.example.com"
63.121.41.178 - - [04/Aug/2003:04:38:13 -0400] "GET / HTTP/1.0" 200 1551 "-" "Mozilla/4.0 (compatible; MSIE 5.0; www.galaxy.com; www.example.com; +http://www.galaxy.com/info/crawler.html)" "www.example.com"
Notice: it includes the URL that it visits in the UA string... strange stuff.
Here is a quote from the their site:
GalaxyBot is a new spider developed by the Internet's first directory provider Galaxy.com. In addition to providing highly relevant directory results, Galaxy.com will soon be offering full-text searches from across the web, searching billions of documents.
Though it does not say how "new" the spider is. That page is not dated or anything...
Seems like we are getting a new search engine to play with :)
[edited by: moltar at 9:50 pm (utc) on Aug. 5, 2003]
The best measure of quality in a search engine is both the amount and quality of traffic you are able to get it to send you.