Welcome to WebmasterWorld Guest from 54.159.250.110

Forum Moderators: DixonJones & mademetop

Message Too Old, No Replies

How do I automate raw logs download

Vertex control panel and daily log...

   
12:38 pm on Oct 15, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I manually click on my vertex control panel "download raw logs" each day.

I wonder how to automate that process (I have 2 sites and each day I must log in and click each link)

Any script? Tool?

I know I can use the server statistics but that's not the point, I also need that files on my pc..

1:32 pm on Oct 15, 2003 (gmt 0)

10+ Year Member



How about FTP?
5:49 pm on Oct 15, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



and how do I schedule that?
the file has a date format like mysite-10-10-2003.gz ...
so everyday has a different name..
6:12 pm on Oct 15, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



There's probably a more elegant way to do it. But since I'm not a programmer, this is what I would do:

1) Create a batch FTP job that retrieves all your .gz files to a local directory.
2) Decompress it (gzip -d).
3) Then use xcopy /y (for DOS) which suppresses prompting to confirm you want to overwrite a file.

If search long enough, you'll find command line tools that you can also use batch rename files or perform some processing for you. Its all very cool what you can find already written by geeks out there.

1:31 pm on Oct 16, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



You need two programs one to build the file to get the logs the other to actually get the logs. This is what I use, it may not be the most elegant or fast but it does work.

use the scheduler to run them, run build_log_gatherer.vbs first then get_yourdomain.bat

I run this

program 1 - build_log_gatherer.vbs



option explicit

'this program gets the log files from an FTP Server

dim build_filename' the name of the file that will need to run from the scheduler
dim build_directory' the location it will be built
dim login_filename' the name of the file with the login info
dim ftp_file' the file we want to get
dim dest_loc' final location of the file that will be ftp'd
dim ftp_site' location to ftp to
dim ftp_directory' ftp directory
dim ftp_username' login username
dim ftp_password' login password
dim temp_str, temp_strII

build_filename= "get_files.bat"
build_directory= "c:/logs/"
login_filename= "get_yourdomain.txt"
ftp_file= ""
dest_loc= "c:/logs/"
ftp_site= "" ' put ftp site here
ftp_directory= ""' put ftp logs here
ftp_username= ""' put username here
ftp_password= ""'put password here
temp_str= ""
temp_strII= ""

call write_blank(build_directory, login_filename)
call write_send(build_directory, login_filename, ftp_username)
call write_send(build_directory, login_filename, ftp_password)
call write_send(build_directory, login_filename, "cd " & ftp_directory)
call write_send(build_directory, login_filename, "get " & log_filename(ftp_file))

call write_blank(build_directory, build_filename)
call write_send(build_directory, build_filename, "cd " & dest_loc)
call write_send(build_directory, build_filename, "ftp -i -s:" & build_directory & login_filename & " " & ftp_site)
call write_send(build_directory, build_filename, "bye " )

function write_send(loc, file, txt)
Dim ofs
Dim objtextstream
Set ofs = CreateObject("Scripting.FileSystemObject")
Set objtextstream = ofs.openTextFile(loc & file,8, True)
objtextstream.writeline txt
objtextstream.Close
Set objtextstream = Nothing
Set ofs = Nothing
end function

function write_blank(loc, file)
dim fso, MyFile
Set fso = CreateObject("Scripting.FileSystemObject")
Set MyFile = fso.CreateTextFile(loc & file, True)
MyFile.Close
end function

function log_filename(ftp_file)
'put the prefix for your files in the first set of quotes in the next line
ftp_file = "" & left(monthname(month(now()-1)), 3)
if day(now()-1) < 10 then ftp_file = ftp_file & "0"
ftp_file = ftp_file & day(now()-1) & "_" & year(now())
log_filename = ftp_file
end function


program 2 get_yourdomain.bat



cd c:/logs/
ftp -i -s:c:/logs/get_yourdomain.txt yourdomain.com
bye