Analog question

Can I see what googlebot crawls?

[Perfection]

Not sure if this has been covered (probably has, I just missed it), but is there a way, using Analog, to see what pages of your site were crawled by Googlebot, and/or other spiders for that matter. If so, how? I'm kind of a newbie with Analog, so try not to use big words. =]

[Birdman]

The Analog version I use does not show that type of info. You may have to download your raw log files for that.

[Glacai]

If you do have your raw logs, add

BROWEXCLUDE *
BROWINCLUDE Googlebot*

to your config file, and run analog.

Or use a seperate config file and tell analog to use that instead.

[Perfection]

Thanks for the replies. I forgot to mention that I AM downloading my raw log files.

Glacai, I used that code and it worked, but it only showed that googlebot visited the site. It doesn't show what specific pages it crawled. Is there anyway to see this?

Thanks again.

[Glacai]

It should show it in the request report.

If you want to see it daily, weekly etc you have to set the FROM and TO dates in your config file.