schibstedsokbot

[bobothecat]

81.93.168.74 - - [25/Mar/2006:07:46:03 -0700] "GET /robots.txt HTTP/1.1" 200 4731 "-" "schibstedsokbot (compatible; Mozilla/5
.0; MSIE 5.0; FAST FreshCrawler 6; +http://www.schibstedsok.no/bot/)"

Did read robots.txt, but banned since the url they provide takes you to a login screen.

[keyplyr]

My guess is just another corporate intranet who has purchased FAST search technology; lots of them around. Personally, I don't mind if employees surf my site during work hours. Most of my sales occur during that time frame :)

[Pfui]

Here [webmasterworld.com]'s my FAST rant, and here are the schibstedsokbot / Schibsted Sok bots I've seen this month...

"FAST Enterprise Crawler 6 used by Schibsted Sok ([e-mail])"

"schibstedsokbot (compatible; Mozilla/5.0; MSIE 5.0; FAST FreshCrawler 6; +http://www.schibstedsok.no/bot/)"

...and their servers:

sch-fast-se-crawl01.dev.osl.basefarm.net
sch-fast-se-crawl02.dev.osl.basefarm.net

sch-fast-se-isearch01.dev.osl.basefarm.net
sch-fast-se-isearch02.dev.osl.basefarm.net

sch-fast-se-crawl01.osl.basefarm.net

All robots.txt-readers but I've never, EVER seen anyone come in from a FAST site per se. (Wonder to whom they sell the data? So you think corporations? Or private parties? Wow. I feel so -- so -- used.)

.
FWIW:
schibstedsok.no -> schibsted.no -> Schibsted [en.wikipedia.org]

[abermir]

Those user-agents are used by Schibsted to crawl the norwegian and Swedish web. (Including .com were pages are found to match the languages in focus).
Schibsted are using FAST technology to power two local search portals (And more to come).

The crawler shipped with FAST is by default identified as "FAST Enterprise Crawler 6 used by X ([e-mail])".
However, "advanced" users can change the user-agent as they like.