Welcome to WebmasterWorld Guest from 54.160.254.203

Forum Moderators: DixonJones & mademetop

Message Too Old, No Replies

Using log files to find how a visitor came to your site

     
8:24 pm on Dec 4, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 14, 2003
posts:1782
votes: 0


I'm wondering if there's anything in the long strings of data in log files that can be used to tell where a visitor came from. Specifically, I subscribe to a shopping website, but I don't trust their conversion reports. I'd like to be able to look at my log files if possible to see where a buyer came from initially.

Is that possible?
11:37 pm on Dec 4, 2012 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month

joined:Apr 9, 2011
posts:12721
votes: 244


If visitors got to your site by clicking a link, logs will tell you where the link was. But if they typed in your address or used a bookmark-- or if their browser doesn't send referers-- logs won't say what site they were on. If that's what you meant.

The referer (their spelling, not mine) is typically the second-to-the-last thing in logs: before the User-Agent, after the pair of numbers showing response code and filesize.
11:57 pm on Dec 4, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member jimbeetle is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Oct 26, 2002
posts:3292
votes: 6


If you have any but a minimum amount of traffic you'll have to use a log analyzer of some sort. Many hosts offer at least one analytics package, some multiple. Check your hosting control panel, you might already have a package installed.

There are also some downloadable desktop packages that, if heavy lifting isn't involved, offer free or lite versions that should do the job.
6:56 am on Dec 5, 2012 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month

joined:Apr 9, 2011
posts:12721
votes: 244


you might already have a package installed

Note that this means analytics-- like Google Analytics or piwik-- not the basic analog stats that are built into most hosting plans. Those are essentially useless for the kind of information you need.
6:59 pm on Dec 5, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 14, 2003
posts:1782
votes: 0


Thanks for the replies. I only need to filter the log files to show the lines for the thanks.asp page. I can create a .txt file of those using the command window.

I've done that already, though, and I don't see any referrers. I don't see any commonality in any numbers in all of the different lines for the thanks.asp page, so I don't know if they came from the same shopping site, if they came in directly, or if they came from Google Adwords.

Adwords isn't showing information in the log files other than simply "Google". No indication as to keywords or phrases.
11:00 pm on Dec 5, 2012 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month

joined:Apr 9, 2011
posts:12721
votes: 244


Post a sample line from raw logs and let's see what it looks like. Obfuscate one block of the IP, and change any complete URLs to "example.com" to prevent auto-linking.
4:46 am on Dec 9, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 14, 2003
posts:1782
votes: 0


Here's one:

2012-12-08 22:05:58 W3SVC1410461454 CLOUD-77ED0817D 21X.3X.1X.170 POST /thanks.asp - 443 - 9X.16X.4X.241 HTTP/1.1 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.55.3 (KHTML, like Gecko) Version/5.1.5 Safari/534.55.3 ASPSESSIONIDQSRTARQD=LJBJJHHBKJMJAFJFOMNKHMHM; __utmb=143293219.4.9.1355003122844; __utmz=143293219.1355003095.3.3.utmgclid=CKC-gfHji7QCFUjZQgodZncACg|utmccn=(not set)|utmcmd=(not set)|utmctr=used%2Acme widget for sale; __utmc=143293219; __utma=143293219.250154279.1355001743.1355002883.1355003095.3 [mysite.com...] www.mysite.com 200 0 0 2843

[edited by: bill at 2:24 am (utc) on Dec 25, 2012]
[edit reason] formatting [/edit]

7:47 am on Dec 9, 2012 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month

joined:Apr 9, 2011
posts:12721
votes: 244


Urk. Please tell me that's That Other Server, not Apache. You will need someone else to do the line-by-line translation.

What on earth are all those cookies doing in the logs? (And, o/t, what's that __utmb and __utmz business? I see it all over the place when I'm cleaning up cookies in my own browser. Got a nasty feeling it's something to do with GA.)

Anyway, it's no use looking at the /thanks page, because the referer for that will always be something on your own site-- most likely a Confirm Purchase type page. You need to look at the visitor's first appearance on your site.
3:05 pm on Dec 9, 2012 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:10544
votes: 8


it looks like the referrer in your example is the cart.asp page.
you want to look at this column for the entry pages referred by the shopping website.
perhaps you can reassemble a visitor's session by working backwards from each thanks.asp log entry.
try sorting your log file by IP, User Agent, Date then Time.
7:40 am on Dec 10, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 14, 2003
posts:1782
votes: 0


Thanks for the help. I thought the search phrase carried through to every page, but I guess not.

That may be GA cookies, but it's not on all pages. I wonder if it might be Adwords.
9:42 am on Dec 10, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Aug 27, 2002
posts:765
votes: 0


Don't know how to program this (should be a few simple lines of php), but in my thinking it should work like this:
a) visitor comes to your site
b) store the search phrase and referer in a cookie
c) visitor goes to merchant site and buys
d) visitor returns to your site's thank-you-page
e) read out the cookie and store the cookie data in a text file
5:52 pm on Dec 10, 2012 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month

joined:Apr 9, 2011
posts:12721
votes: 244


:: belated "D'oh!" moment ::

The originating site must link to some specific page on your site. So that's where you need to check for referers. And then match those up with requests for the /thanks page to see which ones went the whole way from Other Site to a completed purchase.

I thought the search phrase carried through to every page

Nope, logs look only at the immediately preceding page. That's what cookies are good for: long-term information, where "long term" means anything after the first page. If you don't speak php you can also set cookies using mod_rewrite, javascript, and probably several other ways that I don't know about.