FAST MetaWeb Crawler Question

I've never seen it before. What's your experience with it?

[GaryK]

The full user agent is:

FAST MetaWeb Crawler (helpdesk at fastsearch dot com)

WhoIs identifies it as belonging to FAST but I've never seen this incarnation of their typical user agents.

It read/respected robots.txt and took one page every 1-3 minutes.

[GaryK]

Surely someone has seen this user agent string in their logs. Can you confirm for me please if this is a legitimate FAST crawler? I've never seen it in my logs before and other FAST user agent strings look different than this one does. And yet the IP does trace back to whatever it is FAST is an acronym for; I can't recall it now. Thank you.

[wilderness]

Gary,
There's many old threads on Fast. They were acquired a while back by another SE, don't recall who.

There's even a link to a Fast thread in the forum LIBRARY #3.

Don

[volatilegx]

Can't say as I've seen this one yet. The search end of FAST's business was sold to Yahoo! awhile back. Maybe FAST is developing a new meta search engine and is using the bot to spider pages for that. If you have IP addresses for this spider, please post them here :)

[GaryK]

This is the new one I saw in my logs.

66.151.181.12
FAST MetaWeb Crawler (helpdesk at fastsearch dot com)

There have been quite a lot of user agent strings like the one below from FAST where some company name is prefixed to FAST Enterprise Crawler and an e-mail address is appended in parenthesis.

212.48.8.142
Matrix+S.p.A.+-+FAST+Enterprise+Crawler+6+(Unknown+admin+e-mail+address)

Was it AllTheWeb that took over FAST's search business?

[volatilegx]

> Matrix+S.p.A.+-+FAST+Enterprise+Crawler+6+(Unknown+admin+e-mail+address)

This would be a licensed crawler.

> Was it AllTheWeb that took over FAST's search business?

Alltheweb is owned by Yahoo!.