Msg#: 4224442 posted 8:19 am on Oct 31, 2010 (gmt 0)
Mach5 FastStats [mach5.com] has been around awhile but it gets the job done for files up to 1.7 GB (for their $99 paid version). Their free version only works up to 5,000 lines of a log file. Here is a sample report [mach5.com]. On the left hand nav click on the Search Engine Spiders link in the Visitor Information category to see how they display the search spider visits. The software installs on your desktop computer. It's an oldie but a goodie at a reasonable price.
Msg#: 4224442 posted 8:20 am on Oct 31, 2010 (gmt 0)
Thanks for reply Mark.
I use notepad plus too, but for example, recently i found indications that some one was scanning our website with "acunetix_wvs_security Scanner". the scanner was sending some values to a web form. so i discovered it.
If i want to go through logs to get any common mark this scanner is using to block future scans, how can i do it with software's such as TextPad and notepad+!
Msg#: 4224442 posted 5:20 pm on Oct 31, 2010 (gmt 0)
You can use a command line utility called "grep" to pull out and list only lines from your text file that match a pattern you define, and then inspect them for similarities. At least that would filter out all the lines you're not interested in. You can redirect the grep output into a separate text file, which you can then inspect in a text editor. There are free versions of grep for Windows.
For more advanced filtering and sorting, you could import your log data into a MySQL database, which won't have any trouble with 1GB or more of data.