homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

Fast? Crawler

 4:02 am on Dec 3, 2000 (gmt 0)

FAST-WebCrawler/2.2-pre24 (crawler@fast.no;http://www.fast.no/faq/faqfastwebsearch/faqfastwebcrawler.html)
But the IP resolves to:
Host reachable, 120 ms. average -

Exodus Communications Inc. Boston2
2831 Mission College Blvd.
Santa Clara, CA 95112

Center, Network Control CompServ@Exodus.net
1.888.2.Exodus (FAX) 1.888.2.Exodus


Updated: 29-Aug-2000
Source: whois.arin.net

Is this normal?? I thought .no was in Norway?
Also IP



 5:00 am on Dec 3, 2000 (gmt 0)

Yes it is the real thing, they've been coming out of exodus for about 4-5 months now. Many of the major search engines have spiders coming out of there. I did some pricing of their facilities and they are actually very cost effective for a large scale set-up. I'd imagine they are there because it just makes economic sense to do spidering off a U.S. backbone.


 5:58 am on Dec 3, 2000 (gmt 0)

It alarmed me cuz it set off a user agent flag in a script I use and the Ip list has always been pretty up-to-date so far...probly cuz of the info right here in this forum. Thanks fer da info. ( you dont have a napoleanic complex, do you littleman?) :) just kiddin'


 8:23 am on Dec 3, 2000 (gmt 0)

No napoleanic complex here.

You know you should see me! I have to have a stepping stool to make it into my computer chair. But it's cool because I save a fortune in clothes, I've been wearing the same size since I was seven, and kid size shoes are much cheaper then adult's. I always get to pay half price for everything, because I get the twelve and under discounts. Being under three feet tall really has it's advantages.


 3:14 pm on Dec 3, 2000 (gmt 0)

<belly Laugh> Must be nice.I have the opposite problem...the older I get the bigger I seem to be getting. Unfortunately, I used to be 6'4' but now I'm as big around as I am tall. I'm so fat now when I laugh it takes a minute or two for me to "settle" down again. So please, take it easy on me so I don't hurt myself.
Fast is aggressively hitting my domains with new IP's....here they be:
These are new to me anyway. I think I'll celebrate and go eat a box of twinkies and wash it down with a diet coke.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved