Sorting out AOL from the uniques

how many IPs per visitor?

[tedster]

I'm in the process of dealing with AOL's IP addresses in creating some reports. As a rule, AOL assigns multiple dynamic IP addresses to one user/one visit, and this site gets above average number of AOL users.

So I've got a script that pulls out all the AOL hits -- and I'm wondering if there's a rule of thumb for translating that list into a rough estimate for the number of people it represents.

In other words, is it, on average, 8 AOL IP addresses per visitor? Or is it more like 6? And would it be rare or common for AOL to dynamically reassign the same IP to a different visitor within 30 minutes?

[jimbeetle]

Hey tedster,

You got me thinking so I set up a page on a test site and hit it a few times throughout the day through AOL, both DSL and dialup.

DSL 64.12.96.169 10:56:51
DSL 64.12.96.169 11:31:23
DSL 205.188.209.40 12:16:40
DSL 152.163.252.34 12:29:08
DU 205.188.209.40 12:48:55
DU 205.188.209.40 14:06:49
DU 152.163.252.34 14:26:50
DU 152.163.252.34 15:23:40
DU 205.188.209.40 15:39:42
DSL 205.188.209.40 18:25:09
DSL 205.188.209.40 23:22:48

This doesn't help too much and can't really see any way to extrapolate the information even if there is a larger sample size. Wish there were a good rule of thumb as a couple of our sites get very heavy traffic from AOL users.

Jim

[aspdaddy]

Do you not have the user agents in the log?

If you can determine AOL by the IP, I would use UA & Time to sessionize those hits, not perfect as there may be duplicate UA's

[Key_Master]

Here's a different way to calculate it: number of hits from AOL visitors / average page views per visitor

So if your site averages 5 page impressions per unique visitor and you get 1000 hits from AOL IP's, then 1000 page requests / 5 = an estimated 200 unique visitors.

Or use cookies to get a more accurate count.

[tedster]

Thanks for stimulating my burnt out brain.

This is a new client and I'm trying to get a good picture of their traffic for many reasons. One is that they are looking for the most fruitful locations to hold seminars for their interested public and they want to pick cities where they have a good user base.

We're doing some geo-location from IPs and it's fascinating to see how non-homogenous their accesses are. I've advised them to ytack with persistent cookies and session cookies and that's going to happen, but it doesn't help me now with their legacy data.

I like your averaging idea, Key_Master. For this purpose it's probably the easiest approach I've considered so far and it will give me something intelligent to say right away.

aspdaddy, I also like your idea. I'm going to generate a subset of the logs that just has the AOL hits and then see what an analysis on that subset looks like. Looking at the data from both of these approaches should give similar results. If there's too much divergence, then I won't trust either one -- then we'll just have to wait until we have some cookie data.

[piskie]

Tedster
Am I wrong in thinking that AOL dynamicaly assigns an IP address each time the user connects to the internet and that IP address stays for the duration of the session. By that I mean until they disconnect and a different IP address would then be assigned when they connected for the next session.

[tedster]

AOL will assign as many as 8 IP addresses per connection - I see one IP address asking for the page, another asking for an image, a third asking for another image, and all within seconds -- obviously the same user session on a visual inspection of the log.

It's the "as many as" part that is particularly frustrating. If the same IP always asked for the pages, then that would help. But I don't think that's true. And more than one AOL user within the same 30 minute period makes it quite nutty.

Geo-location only reports AOL or AOLUK. No real help there.

[aspdaddy]

I spent a lot of time looking for a way to do this and came to the conclusion that a heuristic approach is probably better.

Even if you understand the characteristics of AOL IPs well, they might change the format, and anonomizers and some networks give similar problems.

Depending on the type of traffic & fields available, counting unique UA's, certain octects of the IP, external referers or even homepage hits can give a rough idea of session numbers.

If you want to reconstruct sessions though, thats much harder.

[claus]

Just dropped by as the subject is interesting although i have none (sic) AOL visitors whatsoever... anyway Key_Master have provided the solution already it seems, unless of course that the site has a majority of AOL users. It's a technique applied in some significant places i know of and it's valid as long as it's used for a residual group and not a large one or the majority.

Apart from that ...cookie tracking. It's always better than logs imho, but that's been mentioned also.

This AOL thing... just wondering... is this weird way of handling privacy their new image? ip-switching and encrypted seachstrings and all...

/claus

[tedster]

Using multiple IP addresses is not at all new for AOL - it's tied to the way their caching servers are set up, I think (another fly in the ointment!)

[claus]

well, i learn new stuff constantly ;) seems to me that AOL is potentially doing..hrm.. not-so-honest-people a favour by pulling these tricks.

[onlineleben]

Just in case you want to build a tracking application for your AOL users, IP Address ranges for Proxys and Cliensts can be found here [webmaster.info.aol.com]

[dkubb]

I believe some proxy servers send through http headers that say which IP address they are proxying for. I don't know how well AOL supports them, but the headers I've heard of are:

- X-Forwarded-For
- Client-IP
- Via

Based on some research, I doubt AOL sends these headers the same way other Proxies do. Most people referred to their proxies as exibiting non-standard behaviour.

Is there anyone with an AOL connection that can test this?

In perl and probably PHP, these variables will be accessible as environment variables called:

- HTTP_X_FORWARDED_FOR
- HTTP_CLIENT_IP
- HTTP_VIA

One other thing I thought of, are you factoring in AOL's caching behaviour? If they have all your content cacheable your logs could show lower numbers of AOL requests vs the true number of requests.

[tedster]

Yes, this is all part of the challenge. Thanks for mentioning the caching factor, dkubb.

The use of multiple IP addresses assigns several uniques to the same user, and caching hides some users. Plus the same IP may be assigned to more than one user.

I'm just hoping I can find a "rule of thumb" that puts me somewhere in the ballpark, and in this case, I'd rather be under the true number than over so I may not try to account for caching.

I understand that all of this is done to give the AOL user a faster surfing experience, and it was very helpful back in the day of slower modems. But, sheesh!

[GuinnessGuy]

Greetings,

Speaking of AOL IP addresses....

I'm attempting to do state by state advertising and I've noticed that when I do this, the AOL IP addresses look very similar when I get a snapshot of who is on my site. For example, say I just did a ad to people in CA, then I went to look at my logs and the AOL addresses were all(something like), 123.456.789.xxx, where only the 'x's varied. Then I went to another state and ran the ad, say, New York and when I checked my log for that time I found a different set of similar IP addresses, as in, 212.343.545.xxx, where only the 'x's varied.

So my question is: Can one tell from the AOL IP's what state someone is from? Or if not a state, perhaps a region?

Any help would be appreciated.

GuinnessGuy

[tedster]

This article from eweek [eweek.com] is two years old. At that time the answer was no, AOL IP addresses cannot be geo-located.

I'm currently using a geolocation service that does pretty well on most non-proxy IPs -- better than 90% are narrowed down to a city or region. But for AOL they can only return country. If this service could do it, I'm pretty sure they would. They update their databases several times a day.

[GuinnessGuy]

Greetings,

Thanks Ted. It's as I thought. It would seem that AOL hands out very similar IP addresses at any given time so my pipedream of having those similar IP addresses having something to do with geo-location were wrong.

If I were AOL, I'd offer a geo-location service for a fee. Seems that they could make some decent money doing this.

GuinnessGuy

[cfx211]

AOL IP's should almost always resolve back to a bunch of server farms in northern VA, Reston and some other town.

I think a lot of AOL's multiple IP addresses has to do with their caching strategy. AOL's proxy servers cache a ton of static pages so you can never really tell how many page views your site truly has because sometimes they are being served from a cached version on an AOL proxy server.

This point is helpful to remember if you see a ton of first visit pages that are a couple of pages deep into your site and there are not a lot of direct links into that page. While there are probably some bookmarks, it is more likely that your home page has been cached, and the first server request is a page or two in.

Finally keep in mind that AOL has hundreds of proxy servers and all of them will show up significantly in your logs so always be looking for new IP ranges if you plan on monitoring IP blocks.