homepage Welcome to WebmasterWorld Guest from 23.21.9.44
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Are these spiders?
scott




msg:396307
 12:59 pm on May 24, 2000 (gmt 0)

I get a bunch of these every day...."cache-ra01.proxy.aol.com".......Are these visitors, spiders, or what?

 

Brett_Tabke




msg:396308
 1:10 pm on May 24, 2000 (gmt 0)

aol caches all data pulled through it's network. those are aol proxy cache spiders. more info (ie5 only).

scott




msg:396309
 4:19 pm on Jun 2, 2000 (gmt 0)

Here's a weird one..... 64.14.36.46 The log says spiders.ny.rubis.net. Anyone ever hear of this one before?

Ludo




msg:396310
 2:26 am on Jun 7, 2000 (gmt 0)

Brett what do you think of this ip's. They are all from AOL, but recently I have seen this more in my website log instead of the one with cache.

spider-tl074.proxy.aol.com
spider-wm023.proxy.aol.com
spider-wi034.proxy.aol.com
spider-wi033.proxy.aol.com

--------------
I just went for trace route quiry for spider-tl074.proxy.aol.com

and this is what I got
----------
Trace Route Results

1 ATHM-216-217-xxx-129.home.net (216.216.3.129) 1.209 ms 1.111 ms 1.062 ms
2 10.252.86.53 (10.252.86.53) 5.856 ms 9.968 ms 5.633 ms
3 10.252.84.1 (10.252.84.1) 5.222 ms 5.273 ms 5.265 ms
4 c1-pos6-1.snjsca1.home.net (24.7.74.61) 5.627 ms 5.619 ms 14.290 ms
5 sjo-brdr-04.inet.qwest.net (205.171.4.85) 6.583 ms 6.461 ms 5.968 ms
6 sjo-core-01.inet.qwest.net (205.171.22.11 7.218 ms 9.237 ms 5.804 ms
7 sfo-core-02.inet.qwest.net (205.171.5.123) 8.198 ms 7.216 ms 8.755 ms
8 jfk-core-02.inet.qwest.net (205.171.5.113) 67.648 ms 67.878 ms 131.864 ms
9 jfk-core-01.inet.qwest.net (205.171.30.1) 71.814 ms 80.494 ms 67.697 ms
10 wdc-core-02.inet.qwest.net (205.171.5.235) 101.629 ms 70.127 ms 69.877 ms
11 wdc-edge-05.inet.qwest.net (205.171.24.86) 78.019 ms 69.990 ms 70.066 ms
12 205.171.54.94 (205.171.54.94) 69.848 ms 69.968 ms 70.083 ms
13 bb2-dcl-P6-1.atdn.net (204.148.99.69) 110.932 ms 109.150 ms *
14 bb2-rtc-P7-3.atdn.net (204.148.97.90) 129.793 ms 136.470 ms 140.230 ms
15 wc2-rtc-P9-0.atdn.net (204.148.101.162) 145.901 ms 131.348 ms 123.932 ms
16 tot2-r8-P4-0-0.aol.com (152.163.132.94) 125.225 ms 130.884 ms 130.806 ms
17 spider-tl074.proxy.aol.com (152.163.207.209) 133.596 ms 138.503 ms 138.306 ms

----------

???

Brett_Tabke




msg:396311
 3:35 am on May 15, 2001 (gmt 0)

Those are just classic AOL proxy cache spiders.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved