Forum Moderators: open

Message Too Old, No Replies

Identify Robots vs Humans?

How to get an accurate visitor count?

         

peterdaly

4:03 pm on Feb 18, 2003 (gmt 0)

10+ Year Member



I am trying to figure out what my conversion ratio is. I have been using the Webalizer visitor count, but I suspect that is not very accurate due to all the bots (some that I want) crawling my site.

Is there a way at the application level I can identify a human visitor vs. a bot? I would be interested in tracking visits based on session creation, but not "count" the visitor if it isn't a human.

Any thoughts on the best way to do this? I know I could create a manual list of bot "signatures" to filter out, but I am hoping there is an easier way.

Moderators, if the post if it is better suited for another forum, please move it to where it is more relevant.

Thanks,

-Pete

brotherhood of LAN

4:09 pm on Feb 18, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Hey Pete, most folks use the forum to identify spiders, so I guess its alright here ;)

This is probably a bit of an over simple solution (bit of a margin of error) but if you add the % of Microsoft IE/Netscape and Opera hits that'll be moost of the human visitors I believe.

wilderness

8:23 pm on Feb 18, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Pete,
Welcome to Webmaster World.

If your using the Webalizer stats provided by a "hosting site" than the stats are generally useless at least IMO.

If you have your won server and you custom configure Webalizer than you'll get ggod stats.

If your paragraph one than the best alternative is to download your logs and use a freeware program named ANALOG.
100 % conigurable.

Don

peterdaly

10:30 pm on Feb 18, 2003 (gmt 0)

10+ Year Member



I have my own dedicated server, with complete control over the logs.

How do you suggest I modify the webalizer config? I am not at all familiar with it beyond basic running against a log file.

-Pete

wilderness

10:44 pm on Feb 18, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Pete,
Neither am I familiar with the Webalizer config. Nor was I familiar with the Analog config until I started reading the options and tinkering.

Analog contains an option to omit counts of bots by either name or IP.
Analog contains an option for omitting or retaining most any count.

From what I recall reading briefly (until I found out I couldn't use the software) in the Webalizer help files was the same diversity as Analog in configuration.

Some of the diversity in Analog includes omitting specific file formats, specific files, including counts down to a single hit rather than just "the top whatever" and many more options.
Even referral counts down to the page.

Accumulate a list of bots and exclude them in the config.
Wish I could be of more help. However there would be no benefit to myself learning learning the Webalizer config which I cannot use.

peterdaly

11:19 pm on Feb 18, 2003 (gmt 0)

10+ Year Member



Thank Wilderness!

That's good enough to get me started...I have a project for tomorrow morning. :)

-Pete