Referer data - Server side Vs. Client Side

Can we get better data using a combined approach?

[mipapage]

Hey all,

A newer site we're working on is getting horrible returns for it's referer data.

We're using a PHP session/cookie thing to get the refere data and stuffing it into mySQL.

I'm wondering if a combined approach of both a Javascript cookie and a PHP cookie would do better?

Am I out to lunch in thinking that this can be done?

---

I tested at home here, and although NIS (norton) munges my referer data, a simple "<a href="javascript:history.go(-1)">Go Back</a>" works just fine. There must be a way to use a cookie with javascript to get the data then pass it to php, no?

[StupidScript]

Whether you use PHP or Javascript (or both) to set cookies, you are accomplishing the same thing with about the same performance characteristics. Both/each take more time than simply writing the data directly into the MySQL db.

1) A visitor lands on your site
2) Grab the referrer (PHP variable HTTP_REFERER)
3) If it is an external referrer, dump it to the db along with the session id and client IP address.

Easy enough to use PHP for this, and since it happens at the server immediately upon visitor entry, and not after the page has been delivered to the client (JS), it is more likely to be accurate, and to not be acted on by "privacy"-type client software.

As you noted, there are some programs out there (i.e. NIS) that intentionally munge or disguise the visitor's history information, perhaps even including the referrer. If you grab the referrer before the request has left the server, you are less likely to have these types of programs impact your data.

How do log analysis programs see your referrer data? Is it equally "horrible"?

[mipapage]

Hey StupidScript -

Thanks for the response!

I'm doing pretty much what you described already - we use php to grab the $_SERVER['HTTP_REFERER'] data and dump it into mySQL.

The problem is that these programs that munge that value (if your have NIS, for example, try a simple 'print_r($_SERVER)' and see the havoc with the firewall enabled, see the data with it disabled).

When I test things out here at home, Javascript seems to be able to 'keep' the referer data, as it doesn't have to pass through the firewall, therefore no munging of data.

What I would like to do, and I imagine others are doing, is have JS, which works client side, put the data into a cookie (also client-side, of course) that I then access with PHP to put the data into mySQL.

Ideally a combination of both would be great, for those with JS disabled.

With regards to "How do log analysis programs see your referrer data?", I have yet to do a proper test, but I would imagine that, as analysis programs work on raw log data, which is server side data, referer data won't be there for the same reasons as mentioned above.

[cfx211]

Does this site have a lot of entries through emails or ppc or anything else that has tracking redirects built into the links? If it does then a lot of those redirects might be blanking out the referrer before the redirect and then there is not a lot you can do about it.

[mipapage]

Hey cfx211,

Thanks for chiming in!

They do use Overture and Adwords, and I've got them appending some Gets onto the url to help out with tracking in that sense.

I'll have to check the logs/db and see if what you say is the case. I'm using something similar to what you laid out in How to track visitors [webmasterworld.com].

[cfx211]

Glad to hear you are using that. Let me know if you come up with anything interesting or run into any problems. I am always looking for new approaches and ways of collecting data.

Does MySQL have a function similar to sys_connect_by_path which is in oracle? If so I can show you how to do path analysis on the logs you write out.

[mipapage]

Thanks,

I'll check, I haven't gotten that far yet. I'm collecting a lot of data, and have been scrambling these last few days to sort out my tables etc. to collect the appropriate data in the appropriate places...

Fun stuff, this is!

[roitracker]

Your referer log data is never going to be 100% accurate. For more info, message 38 of the "blank referers" thread is well worth a read:
[webmasterworld.com...]