homepage Welcome to WebmasterWorld Guest from 54.234.147.84
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Fast? Crawler
Strange
BoneHeadicus




msg:408839
 4:02 am on Dec 3, 2000 (gmt 0)

FAST-WebCrawler/2.2-pre24 (crawler@fast.no;http://www.fast.no/faq/faqfastwebsearch/faqfastwebcrawler.html) 209.202.148.34
But the IP resolves to:

209.202.148.34
crawler24.bos2.fast-search.net
Host reachable, 120 ms. average

209.202.128.0 - 209.202.191.255

Exodus Communications Inc. Boston2
2831 Mission College Blvd.
Santa Clara, CA 95112

Center, Network Control CompServ@Exodus.net
1.888.2.Exodus (FAX) 1.888.2.Exodus

NS.EXODUS.NET 206.79.230.10
NS2.EXODUS.NET 207.82.198.150

EC21-2
Updated: 29-Aug-2000
Source: whois.arin.net

Is this normal?? I thought .no was in Norway?
Also IP 209.202.134.12

 

littleman




msg:408840
 5:00 am on Dec 3, 2000 (gmt 0)

Yes it is the real thing, they've been coming out of exodus for about 4-5 months now. Many of the major search engines have spiders coming out of there. I did some pricing of their facilities and they are actually very cost effective for a large scale set-up. I'd imagine they are there because it just makes economic sense to do spidering off a U.S. backbone.

BoneHeadicus




msg:408841
 5:58 am on Dec 3, 2000 (gmt 0)

It alarmed me cuz it set off a user agent flag in a script I use and the Ip list has always been pretty up-to-date so far...probly cuz of the info right here in this forum. Thanks fer da info. ( you dont have a napoleanic complex, do you littleman?) :) just kiddin'

littleman




msg:408842
 8:23 am on Dec 3, 2000 (gmt 0)

No napoleanic complex here.

You know you should see me! I have to have a stepping stool to make it into my computer chair. But it's cool because I save a fortune in clothes, I've been wearing the same size since I was seven, and kid size shoes are much cheaper then adult's. I always get to pay half price for everything, because I get the twelve and under discounts. Being under three feet tall really has it's advantages.

BoneHeadicus




msg:408843
 3:14 pm on Dec 3, 2000 (gmt 0)

<belly Laugh> Must be nice.I have the opposite problem...the older I get the bigger I seem to be getting. Unfortunately, I used to be 6'4' but now I'm as big around as I am tall. I'm so fat now when I laugh it takes a minute or two for me to "settle" down again. So please, take it easy on me so I don't hurt myself.
Fast is aggressively hitting my domains with new IP's....here they be:
209.202.148.21
209.202.148.31
209.202.148.27
209.202.148.34
209.202.148.12
These are new to me anyway. I think I'll celebrate and go eat a box of twinkies and wash it down with a diet coke.
Peace

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved