Investigative Searches in log files

[sdani]

do you use any software to do investigative searches in web server log files (keyword), bots, specific IP address etc?

Any recommendations?

[oxbaker]

most web analyzer software does this, allows you to filter out for certain variables. Products like webtrends and ias log analyzer will do that for you, or you can dump the logs into a database and do SQL queries on them.

hth,
mcm

[gregbo]

If you know some programming languages like Perl or Python, you can use those to do investigative searches.

[dragan djapic]

Web Log Storming is designed for that purpose. It reads all log files into memory and allows you to dig into any session and to see exact hits on pages, files, images... Downside is that you can't load too much data because of memory requirement.

[jatar_k]

I've always used grep

if there are things that need examination on a regular basis then I use custom logging/logs so that I can tack some individual metrics