Using log files to find how a visitor came to your site
8:24 pm on Dec 4, 2012 (gmt 0)
I'm wondering if there's anything in the long strings of data in log files that can be used to tell where a visitor came from. Specifically, I subscribe to a shopping website, but I don't trust their conversion reports. I'd like to be able to look at my log files if possible to see where a buyer came from initially.
Is that possible?
11:37 pm on Dec 4, 2012 (gmt 0)
If visitors got to your site by clicking a link, logs will tell you where the link was. But if they typed in your address or used a bookmark-- or if their browser doesn't send referers-- logs won't say what site they were on. If that's what you meant.
The referer (their spelling, not mine) is typically the second-to-the-last thing in logs: before the User-Agent, after the pair of numbers showing response code and filesize.
11:57 pm on Dec 4, 2012 (gmt 0)
If you have any but a minimum amount of traffic you'll have to use a log analyzer of some sort. Many hosts offer at least one analytics package, some multiple. Check your hosting control panel, you might already have a package installed.
There are also some downloadable desktop packages that, if heavy lifting isn't involved, offer free or lite versions that should do the job.
6:56 am on Dec 5, 2012 (gmt 0)
you might already have a package installed
Note that this means analytics-- like Google Analytics or piwik-- not the basic analog stats that are built into most hosting plans. Those are essentially useless for the kind of information you need.
6:59 pm on Dec 5, 2012 (gmt 0)
Thanks for the replies. I only need to filter the log files to show the lines for the thanks.asp page. I can create a .txt file of those using the command window.
I've done that already, though, and I don't see any referrers. I don't see any commonality in any numbers in all of the different lines for the thanks.asp page, so I don't know if they came from the same shopping site, if they came in directly, or if they came from Google Adwords.
Adwords isn't showing information in the log files other than simply "Google". No indication as to keywords or phrases.
11:00 pm on Dec 5, 2012 (gmt 0)
Post a sample line from raw logs and let's see what it looks like. Obfuscate one block of the IP, and change any complete URLs to "example.com" to prevent auto-linking.
4:46 am on Dec 9, 2012 (gmt 0)
2012-12-08 22:05:58 W3SVC1410461454 CLOUD-77ED0817D 21X.3X.1X.170 POST /thanks.asp - 443 - 9X.16X.4X.241 HTTP/1.1 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.55.3 (KHTML, like Gecko) Version/5.1.5 Safari/534.55.3 ASPSESSIONIDQSRTARQD=LJBJJHHBKJMJAFJFOMNKHMHM; __utmb=143293126.96.36.1995003122844; __utmz=143293219.1355003095.3.3.utmgclid=CKC-gfHji7QCFUjZQgodZncACg|utmccn=(not set)|utmcmd=(not set)|utmctr=used%2Acme widget for sale; __utmc=143293219; __utma=143293219.250154279.1355001743.1355002883.1355003095.3 [mysite.com...] www.mysite.com 200 0 0 2843
[edited by: bill at 2:24 am (utc) on Dec 25, 2012] [edit reason] formatting [/edit]
7:47 am on Dec 9, 2012 (gmt 0)
Urk. Please tell me that's That Other Server, not Apache. You will need someone else to do the line-by-line translation.
What on earth are all those cookies doing in the logs? (And, o/t, what's that __utmb and __utmz business? I see it all over the place when I'm cleaning up cookies in my own browser. Got a nasty feeling it's something to do with GA.)
Anyway, it's no use looking at the /thanks page, because the referer for that will always be something on your own site-- most likely a Confirm Purchase type page. You need to look at the visitor's first appearance on your site.
3:05 pm on Dec 9, 2012 (gmt 0)
it looks like the referrer in your example is the cart.asp page. you want to look at this column for the entry pages referred by the shopping website. perhaps you can reassemble a visitor's session by working backwards from each thanks.asp log entry. try sorting your log file by IP, User Agent, Date then Time.
7:40 am on Dec 10, 2012 (gmt 0)
Thanks for the help. I thought the search phrase carried through to every page, but I guess not.
That may be GA cookies, but it's not on all pages. I wonder if it might be Adwords.
9:42 am on Dec 10, 2012 (gmt 0)
Don't know how to program this (should be a few simple lines of php), but in my thinking it should work like this: a) visitor comes to your site b) store the search phrase and referer in a cookie c) visitor goes to merchant site and buys d) visitor returns to your site's thank-you-page e) read out the cookie and store the cookie data in a text file
5:52 pm on Dec 10, 2012 (gmt 0)
:: belated "D'oh!" moment ::
The originating site must link to some specific page on your site. So that's where you need to check for referers. And then match those up with requests for the /thanks page to see which ones went the whole way from Other Site to a completed purchase.
I thought the search phrase carried through to every page