AOL question?

cache-mtc-****.proxy.aol.com

[RussellC]

I have asked a similar question before but never got a definite answer. I am seeing this:

spider-mtc-ti034.proxy.aol.com

OK fine a rarely get these in my logs, but if that is some kind of spider, what is this:

cache-mtc-am01.proxy.aol.com
cache-mtc-am07.proxy.aol.com
cache-mtc-ak02.proxy.aol.com

I get tons of these in my logs daily. Different variations, they make several visits to my site each. Anyone have any ideas what these are? Are they IP's from actual AOL users or something from AOL itself?

Thanks

[seth_wilde]

I believe that's AOL's caching system... not a spider..

[RussellC]

Why do they hit my site hard every day though? does it do this on most sites?

[seth_wilde]

"Why do they hit my site hard every day though"

That means your site is popular with AOL users :) Instead of directly serving the user with a page, AOL's cache server requests the pages (often using a different IP address for every file requested). AOL stores different cached copies of files for different users agents and clears the cache every 24 hours..

This process can play havoc on non cookie based stats by inflating your unique user numbers and ruining your pages per visitor stats.. (As breifly discussed in the urchin thread earlier today)

[RussellC]

Ahhh Thank you. I understand now. Yes my host is UNIX/Apache so I get my logs in raw text format and I use Summary to process the logs. It does affect my logs when I have alot of AOL users on the site. How can I filter these out of my logs? Thanks again.

[seth_wilde]

I haven't heard of a good filtering solution (it appears pretty random). You generally have to live with it or switch stats.