|What is in Herndon, VA, USA|
Question regarding log files
| 12:05 am on Dec 28, 2003 (gmt 0)|
I was perusing my log files today and I noticed that Herndon, Virginia, United States has accessed my site double the amount of times as my local traffic. Any idea what is there?
| 12:16 am on Dec 28, 2003 (gmt 0)|
| 12:57 am on Dec 28, 2003 (gmt 0)|
Oic.... why would they be spidering (assuming) my site so much. I am in a different country not really offering too much to the American consumer?
| 3:17 am on Dec 28, 2003 (gmt 0)|
AOL/Time Warner provides internet service not only for those in the US (24.7M) but also overseas -- 6.2M in Europe and another 2.5M in other countries (Argentina, Australia, Brazil, Canada, Japan, Mexico).
AOL is not the only corp located in that area. A lot of companies moved there with some setting up their corporate HQ's. That county contains the highest population in the whole state of Virgina.
You should be able to look up the IP(s) to verify whether it/they belong to AOL. I'm not sure they even have a spider. Google powers their search engine. AOL does cache pages but I wouldn't consider that real spidering. As far as I've seen so far, they've only cached pages which their users visited. It could easily be someone just sitting at home sucking up your site. I've had a couple from Northern Virgina try to do that to mine.
| 9:08 pm on Dec 28, 2003 (gmt 0)|
AOL also uses a ton of rotating proxy machines, making it nearly impossible to track any one user. One particular IP address may in fact be many different people, using different browsers, at different times. Depending on what you are using to view your logs, this may initially drive you to the incorrect conclusion that it is an erratic spider. It also makes it almost impossible to block any single AOL user with a spoofed UA.