WebTrends Pro 5.5b Help

Tracking Google

[netguy]

I have the old WebTrends 5.5b version that I just reloaded for several of my websites. I called WebTrends and there is no longer any support/updates for this Pro version (now they have a whole new line of software offerings that I can't figure out what all they are trying to do, I just know instead of $500, it is now over $3,000).

My question is about 'seeing' Google visits.

Instead of putting the Google information into the "visiting spiders" category, Google is always shown under "Top Browsers."

And instead of detailed IPs and Crawlx, Crawler information, I only see: Googlebot/2.1 ( [googlebot.com...]

Is this a configuration problem, or does this version simply not support Google well?

Thank you for any input.

Steve

[pendanticist]

Steve, let me say upfront, I know nothing about WT.

Forgive me for straying somewhat from the specificity of your post, but there is one point I wanted to clarify in that UA string you've posted.

That isn't the complete string, is it? I'm asking because what you've noted is devoid of a certain + sign as noted below.

From my log files.

crawler11.googlebot.com - - [07/Apr/2003:06:40:49 -0700] "GET /GeoScience.html HTTP/1.0" 200 19418 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

Another poster (UK) and myself (US) are trying to find out why this occurs because their logs do not have the + on any Googlebot visits.

Now I'm on an Apache (they IIS) and every Googlebot visit I've (just about) ever had, contained that + plus sign. Literally.

Do your files show a + plus sign? If so/not, what server are you running on?

Aside from the geographical diffences between UK & US (which I don't think has any bearing on this situation), we were speculating if the lack of the + plus sign may possibly denote a spoofed UA, or is the + sign stripped out by the UK servers somehow? For that matter, whether my server adds the + sign is also a possiblity. <shrug> Dunno.

If you can't speak to this, no biggie. At least your out of 'Unanswered' now.

Pendanticist.

[netguy]

<Do your files show a + plus sign? If so/not, what server are you running on?>

pendanticist... I'm running Apache as well (dedicated server in the U.S.). And what I listed is cut and pasted directly from my lastest WebTrends report - without the "+" sign.

I would LOVE to have the level of detail you have in your logs! Unfortunately, I beleive WebTrends just provides a 'snapshot' of what it going on, without providing the more technical information.

As much as I like WT for most of its reporting, I may have to look at another package to pull more detail from my logs.

Thanks for responding pendanticist!

Steve

[pendanticist]

pendanticist... I'm running Apache as well (dedicated server in the U.S.). And what I listed is cut and pasted directly from my lastest WebTrends report - without the "+" sign.

Ok, Thanks.

I would LOVE to have the level of detail you have in your logs! Unfortunately, I beleive WebTrends just provides a 'snapshot' of what it going on, without providing the more technical information.

I've actually never even looked into those packages as I examine my access_log files manually and frequently. I find an 'anlysis package' lacks immediacy. By that I mean, all the results are 'after the fact' which is too late for me.

As much as I like WT for most of its reporting, I may have to look at another package to pull more detail from my logs.

If you have access to those access_log files and you feel particularly energetic, try examining them manually for awhile and compare them to your WT results before you make any switch. That way you'll enhance your knowledge of WT's intricacies (as well as your log files) and be better versed on how these packages can work for you. What you want, don't want and the like.

Thanks for responding pendanticist!

My pleasure. :)

Pendanticist.

[graywolf]

I recently upgraded to Webtrends Reporting Center from version 5 and saw the same thing you did about google being a browser type not a spider. I tried monkeying a little and was never able to figure out why or solve the problem. Eventually I just set a seperate report for the googlebot to watch it, and gave it it's own exclude/include filter.

Maybe it's "feature" not a "bug" ;-)

[netguy]

<If you have access to those access_log files and you feel particularly energetic>

pendanticist... You've helped me see a whole new set of problems. For the past couple years I have been using WebTrends Live because I didn't want to mess with the reports. Then, to save $35 bucks a site, I had my system administrater add logs to all the domains on both my servers, and use WebTrends 5.5 that I bought, but never got around to using.

SOOO, I just downloaded one of my access_log files to take a look at it - 20 minutes later on a cable connection, I see I have a 420 MEG FILE (and its only been added since February 15 - less than 2 months ago!).

With 3 9GB hard drives on the larger server, and dozens of sites, I'm going to run out of disk space!

I'm obviously missing something here, but there is no way I can keep accumulating these size files.

I think the details of GoogleBot are the least of my worries right now. Going to have to call my sys admin guy in the morning and see what this is all about.

Steve

PS: No wonder WebTrends was grinding and grinding for 30 minutes at a time to generate my twice-a-day reports!

[pendanticist]

>pendanticist... You've helped me see a whole new set of problems.

Uh, ohhh.

I hope it turns out that the solutions those new problems are constructive.

No problem on the GB thing.

Pendanticist.