Forum Moderators: DixonJones
The end result of what I need is to have a file created on demand that lists the referral domain names for a period (the last week is my preference, in highest to lowest order of hits). I would then take this file and include it in a PHP page (my blog index page) for the purposes of listing referring sites. I'd run this script to create a new file each week.
I have found many tracking programs that provide lots of data in charts and lists, but have not found any that will give me this simple list.
I do have full logs with referrer info on my Apache server, MySQL and PHP available. (Also have Analog installed).
A program that reads the log files would work, or a program for which I have to place some code into my main pages to log hits (if it can handle tens to hundreds of thousands of hits a day) is okay to. Or, on that subject, even a hosted service would work too, if that's the only option too.
Any ideas?
awk -F[\"] '{print $4}' yourdomain-combined_log \
¦ sed 's/\?.*$/\1/' \
¦ sort ¦ uniq -c ¦ sort \
¦ grep -v yourdomain.com > somefile.txt
This will read an entire log file (or a list of log files) so you might need to do some pre-parsing to focus on one week - or set your logs to rotate weekly.
When you get to the end you should find that a file is created containing the information you want.
You'll need to replace yourdomain-combined_log with the path to your log file/s.
You can put the command into a shell script and run it using cron - or call it from PHP using system or exec.
Hope this helps
# You can limit the dates analysed with FROM and/or TO: see docs/include.html
FROM 040101
# FROM -00-00-01 [yesterday]
TO 040107
But there is one more step I need to be able to do. Some of the "referrers" don't really have a referral link set up for me--someone might have mentioned my site in their discussion forum for example. Some are search engines. What I need is a program that can read through the Word, Excel or ASCII file which contains one domain URL per line, and somehow tell me which sites do not have a link, so I can remove them from the list.
I've looked around and cannot find anything like this. There are plenty of referring sites link checking programs but not anything that does something as simple as I'm trying to do.
Does anyone have any suggestions?