Customizing Apache logs for referrals

Referrals Problem for Mozilla

[kadimulam]

Hi,

I need your help for the following situation:

Suppose that there is some banner ad say(abc.swf) on www.abc.com/jsp/bannerpage.jsp pointing to an external website www.pqr.com. And the user uses four different browsers say Internet Explorer,Mozilla Firefox,Opera and Safari.

When the user clicks on the banner ad using the above four browsers,Apache logs the following for referrals in the log file.

Firefox,Opera,Safari
www.abc.com/jsp/bannerpage.jsp

Internet Explorer
www.abc.com/jsp/abc.swf

I am using the foolwoing log format:

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" \"%{Cookie}i\"" special

Why does the Apache capture 'www.abc.com/jsp/bannerpage.jsp' for Mozilla, Opera and Safari display and 'www.abc.com/jsp/abc.swf' for IE.

I need some help in customizing the logs or browsers(using java settings in my JSP) so that the information is logged as 'www.abc.com/jsp/abc.swf' for all the four browsers.

Please help.

Thanks,
Kiran

[jdMorgan]

It is not that Apache is logging differently, it is that IE is sending a different referrer. This is probably due to the fact that IE uses an ActiveX control (e.g. C:\windows\system32\Macromed\Flash\Flash9f.ocx) to implement the flash player function, while all the others use a Netscape-style plug-in (e.g. C:\Program Files\Opera\program\plugins\npswf32.dll) to implement flash.

I'm afraid there is nothing you can do server-side to change the logged referrer -- Apache will log whatever the client sends in the "Referer" header of the HTTP request.

You might be able to post-process your logs with a script to replace the IE referrer with a Netscape-style referer (based on a lookup list) before feeding that log data to your stats program, but that's a bit beyond the scope of this forum.

Jim

[kadimulam]

Thanks Jim for the quick reply.

I need to customize logs that represent IE style of referrer(ie 'www.abc.com/jsp/abc.swf') and not Netscape style.Is it possible?

Also, when an image is clicked on a page all the four browsers log 'www.abc.com/jsp/bannerpage.jsp' for the referer field. What i need is a referrer that displays the image file 'www.abc.com/jsp/pgr.jpg'. Can this be done?

Please help.

Thanks,
Kiran

[jdMorgan]

Yes, as I stated, this is a browser behavior issue. You can post-process your log files to look up one "style" of referrer and modify it to another. But this is a scripting problem, not an Apache problem. Again -- Apache will log whatever the browser sends as the Refererer. If the different browsers send inconsistent referrers, there is nothing you can change in Apache to change what is logged.

Jim

[kadimulam]

Thanks Jim

I will try to write a script to change the referral field value in the log file. But i think this is a lengthy process because i need to search for all the referral fields and my log file is around 3GB in size.

What do you say?

One more help:-
This is regarding the logging of IP address.

(%h)
This is the IP address of the client (remote host) which made the request to the server. If HostnameLookups is set to On, then the server will try to determine the hostname and log it in place of the IP address. However, this configuration is not recommended since it can significantly slow the server. Instead, it is best to use a log post-processor such as logresolve to determine the hostnames. The IP address reported here is not necessarily the address of the machine at which the user is sitting. If a proxy server exists between the user and the server, this address will be the address of the proxy, rather than the originating machine.

I got the below information from here [httpd.apache.org...]

Apache is logging the same IP Address for all the users from my office and not the individual IP addresses. Can we fix this?

I have seen one more option %a. Will this work?

Please help.

Thanks,
Kiran

[jdMorgan]

Again, Apache will log whatever it receives from the client. If your office is on a local area network - and individual machines do not connect directly to the internet, then Apache will never receive LAN addresses in requests. LAN addresses are unroutable by definition, which is what makes LANs possible.

Apache will receive the "public" IP address of your office router (as assigned by your ISP), and that is what it will log.

If your server is also on the same LAN as your office workstations, you could re-configure your network, so that all workers access the server using its local address and/or different domain name. This would require you to create and maintain an entry in the "hosts" file of each workstation, to point the local domain name to the local network server address.

Jim