Welcome to WebmasterWorld Guest from 54.162.164.86

Forum Moderators: DixonJones & mademetop

Message Too Old, No Replies

Does Anyone ever watch their stats live?

Use the tail -f command in *nix to watch your stats

     
2:59 pm on Mar 9, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 6, 2003
posts:109
votes: 0


I've spent the past couple of days watching my traffic in real-time using the "tail -f logfile.log" command. I watch where people are coming from and where I rank based on referrer. Maybe this topic has already been covered but I find it entertaining.
3:10 pm on Mar 9, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 11, 2002
posts:2024
votes: 0


Yes. Here's another weirdo...

You might want to take a look into "Who's On" by Parker Software. It's basically a graphic interface to who visits your site(s) right now, including paths and repeat visits and some basic tracking and statistics.

Its geo-targeting is a bit buggy right now and I give Steve a hard time on this, but the software itself is reasonably priced and gives you a very good overview of what's happening.

3:28 pm on Mar 9, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 6, 2003
posts:109
votes: 0


pmkpmk,
I'm glad to here there's someone else that finds real-time stats entertaining. I checked out that software but it's only Win compatible :( I run OS X and SuSE.
3:49 pm on Mar 9, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:June 25, 2002
posts:776
votes: 0


I have a friend who runs a crontab tailing his logs. It also pipes the results through various filters and automatically emails him when it finds something of interest.

Seems a bit nerdy to me.

4:00 pm on Mar 9, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Oct 23, 2004
posts:113
votes: 0


I do something like the above where I stream the logs into a bunch of filters.

Why?
-I know exactly when, where, and what a spider is doing on my sites :)
-Lets me know if there's a error on the site at all when pages are loading

Works very well for me :)

11:03 pm on Mar 11, 2005 (gmt 0)

Preferred Member

10+ Year Member

joined:Aug 2, 2004
posts:551
votes: 0


You sound like that guy on the Nebacanezzer who watches the matrix through the streaming code.
12:44 pm on Mar 12, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 11, 2002
posts:2024
votes: 0


Actually it is a bit like that...

Hey, did anyone ever watched a live tcpdump stream? It's in a way hypnotizing...

12:54 pm on Mar 12, 2005 (gmt 0)

Senior Member from FR 

WebmasterWorld Senior Member leosghost is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Feb 15, 2004
posts:6717
votes: 230


You all may be in need of getting out a lttle more ;-)
4:05 pm on Mar 12, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 6, 2003
posts:109
votes: 0


tcpdump stream... yeah! It still amazes me to see visitors in real-time. Get out more..... this is as entertaining as it gets.
6:05 pm on Mar 12, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Aug 11, 2004
posts:1014
votes: 0


You all may be in need of getting out a lttle more ;-)

By the time I got to this I was thinking much the same!

Yes I have watched the access in real time, but only when analysing a DoS attack.

Matt

6:14 pm on Mar 12, 2005 (gmt 0)

Administrator

WebmasterWorld Administrator jatar_k is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:July 24, 2001
posts:15756
votes: 0


gotta love tail'ing logs

watchin mail bounce around, apache logs, all of it, we use syslog a fair bit and even that is interesting.

I have watched tcpdump's too, fascinating stuff.

The benefit of all this is that it gives you a better understanding of what data is acquired and helps you see what all of your stats analysis programs are actually interpreting.

I would suggest watching logs as a basic excercise for any person who works with web sites.

9:30 pm on Mar 13, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 13, 2005
posts:80
votes: 0


I have tried “Who's On" by Parker Software. You can see exactly what is going on your site in real time. The software looks very nice but still a bit buggy.

Does anybody know of a similar product but which is more stable?

10:37 am on Mar 14, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 11, 2002
posts:2024
votes: 0


Unfortunately no. That's why I try to bug Steve Parker to get moving :-)

As for "getting out more often". Actually I do get out quite often, having family with two little rascals who prevent any form of lazyness. Nevertheless, watching live stats is quite interesting, hypnotizing and - in a way - soothing,

2:41 pm on Mar 14, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:June 9, 2003
posts:1471
votes: 0


Do you lot every tie an old cine film from one washing line to another, look down a toilet paper tube at the film and run along to the end?

...just checking sanity levels.

2:45 pm on Mar 14, 2005 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member pageoneresults is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 27, 2001
posts:12172
votes: 61


Does anybody know of a similar product but which is more stable?

I'm testing a product right now called VisitorVille. It's SimCity for tracking and logging. Really cool stuff. ;)

Testing stability issues now and over the next few weeks.

2:51 pm on Mar 14, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 11, 2002
posts:2024
votes: 0


VisitorVille is cool, but I doubt that one will continue using it once a few weeks have passed. It's more appealing to the toy/gizmo lover who has room in his budget for software. I assume it collects dust once the novelty wears off. I could very well imagine having it running on a seperate screen and ocassionaly glimpsing at it. But for me it has no real use.

Do me a favor, take a look at Who's On at let me know which of the two you still use after a few weeks.

3:06 pm on Mar 14, 2005 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member pageoneresults is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 27, 2001
posts:12172
votes: 61


Do me a favor, take a look at Who's On at let me know which of the two you still use after a few weeks.

No need to do the "few weeks" thing. I've viewed the features list and I don't see a whole lot there. It's a very simple interface that shows you who's on the site.

<added> There are more features there, they are just buried a bit. Haven't done a full view under the hood.

What really appealed to me when first considering VV was the fact that the VV code is valid out of the box, no tinkering to escape characters, etc. It is also very lightweight.

I'll agree that the novelty of it wears off after a bit. But, when used together with the Live Help feature, the novelty becomes an integral part of the process.

I'm sitting here now looking at a city that has over 800 buildings. Each building represents a page on the site. There are buses carrying search engine visitors all over the site right now. There are police cruisers, fire trucks, taxi-cabs and a bunch of people just bustling about.

I can right click a bus and see the passengers. I can click on one of the passengers and see their profile which gives me all sorts of information about their system, IP address, etc.

It's basically a gaming package for your stats! :)

3:11 pm on Mar 14, 2005 (gmt 0)

New User

10+ Year Member

joined:Mar 14, 2005
posts:11
votes: 0


I did this last night with tail -f -s 0.1 *.log for the first time, and I was amazed at exactly how much my servers were serving.
3:15 pm on Mar 14, 2005 (gmt 0)

Preferred Member

10+ Year Member

joined:Jan 28, 2005
posts:614
votes: 0


pageoneresults-

That sounds like a fun little program, I really must check it out. Yes, I can see where the novelty would wear off to a certain extent (just like the SIM games) but what a great way to get a quick view of the organics of your site function.

Freq---

7:52 pm on Mar 14, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 11, 2002
posts:2024
votes: 0


Problem is, you HAVE to watch it! Unlike a stats package, it's LIVE. If you miss something, you missed it.

pageone: I forgot about the Chat feature. That's actually how I got "hooked" by all this. We used (and still use) LivePerson (HumanClick) for Chat for many years already (actually we used it already back when the basic service was free of charge). We implemented it for chat, but it had sort of a live view as well, and that was great for checking results of email blasts, news releases etc or just for seeing referring sites shift.

Being basically not happy with LivePerson anymore (price/value ratio not right anymore), I started to look for something else. But all other packages I found did not have the live-view feature. That's how I ended up with Who'sOn (which incidentally has a chat feature as well which we don't use).

VisitorVille survived for 2 days on my machine: I got too distracted, my co-workers got too distracted, and it gave no real benefit. Plus I found it rather high priced.

8:51 pm on Mar 14, 2005 (gmt 0)

Preferred Member

10+ Year Member

joined:Jan 28, 2005
posts:614
votes: 0


It certainly does seem a little spendy compared to other packages I've looked at. Granted they had much less entertainment value.

I don't think you are stuck with "live" only thought. It looked to me like you can select a timeperiod and play the "stats" whenever you want.

I don't know if I'll end up using it but I may try it out for a month just to get a feel for it.

Freq---

11:24 pm on Mar 14, 2005 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member pageoneresults is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 27, 2001
posts:12172
votes: 61


Problem is, you HAVE to watch it! Unlike a stats package, it's LIVE. If you miss something, you missed it.

Not with VV. There is a VCR feature and you can playback whatever you like. ;)

I don't think you are stuck with "live" only thought. It looked to me like you can select a timeperiod and play the "stats" whenever you want.

You are correct, you can replay the stats. There is both a hosted version and a version you can use to read your log files. It works either way.

Pricing starts at $4.95 USD per month. Not too pricey. It is pretty much inline with most other live stats programs when you get into the larger traffic numbers. The pricing plan is also based on unique visitors and not page views.

2:29 pm on Mar 15, 2005 (gmt 0)

Preferred Member

10+ Year Member

joined:Jan 28, 2005
posts:614
votes: 0


I wish they had a one time purchase option just for looking at the log files. I don't really care if I see the stats "live" but I'd love the interface for replaying the logs for the past week or month. That doesn't seem to be an option.

Freq---

4:05 pm on Mar 15, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 11, 2002
posts:2024
votes: 0


Instead of hosted, you can go for an installabla package. That's actually the (rather high priced) one I had in mind.
8:12 pm on Mar 16, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 12, 2003
posts:1088
votes: 4


Gee if you have that much free time on your hands, why not write some content instead :)
9:49 pm on Mar 16, 2005 (gmt 0)

Administrator from US 

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 25, 2005
posts:14664
votes: 99


I use tail all the time when some twit starts offloading my pages to china, norway or somewhere.

As a matter of fact some idiot tried it at 1am PST the other night.

He overloaded my server being greedy grabbing too many pages simultaneously, so the server sent me an alarm and he was blacklisted inside of 10 minutes of the start of the download attempt thanks to tail :)

10:42 pm on Mar 17, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Oct 23, 2002
posts:48
votes: 0


I took a look at whoson, but I don't have access to my live logs via ftp. I do via scp, it would be nice if it had that option. Hopefully they will add that as an option one day.
I used this command to watch the live visitors without all the image files showing up, works good for me: tail -f logfile ¦grep -iv ".jpg\¦.gif"
11:15 pm on Mar 17, 2005 (gmt 0)

Preferred Member

10+ Year Member

joined:Dec 8, 2003
posts:548
votes: 0


Yes, me too!