One line ftp command

for cronjob accesslog download

[Jaze]

Hi ya all,

First time new topic starter.

I have a web server who doesn't archive the log files so each week I have to download the access log. Being busy, one tends to forget to do this now and then so thought setting up a cronjob was the best way to go about it.

Using MacOS X Jaguar, I found Cronnix to allow me to add in the cron job easily but I'm struggling a wee bit with the syntax for an ftp command.

Each week the log file changes name access-log-1020.gz this week and will be access-log-1027.gz next week so was thinking wild cards?

I'm able to use a one line command to get the current accesslog with:
ftp ftp://user:[e-mail]/%2Fvar/log/httpd/archive/some-domain-access-log.1020.gz ; type=I

but won't allow me to wildcard the date in the access log eg some-domain-access-log.*.gz nor using .?.gz

I've come across mget but I'm not sure how to work it so that I'm still only using one line.

Any suggestions?

[sun818]

Can you have the cronjob reference a shell script?

include this in the beginning of the shell script:

MDATE=`date "+%m/%d"`

Replace your month/day with $MDATE.

ftp ftp://user:[e-mail]/%2Fvar/log/httpd/archive/some-domain-access-log.$MDATE.gz ; type=I

As long as the cron job runs on the day the log is generated, it should pull the correct file down.

[Jaze]

I see what MDATE=`date "+%m/%d"` does and sounds like a good idea but I'm not sure what their routine is. If it is every 7 days (ie every Sunday?) that that would be fine.

I see that cron (via Cronnix) does allow processing of shell scripts which then eliminates the need for one line and also means I should be able to use mget somehow.

I now have a couple of options - 1. Contact the web server and see if they will tell me their routine - therefore running the cronjob (shell script) on that particular day of the week with the MDATE as the date given or

use figure out how to use mget since I can now run ftp commands one by one with the shell script.

Thanks sun818!

[sun818]

mget *.* will prompt you if you want to download each file in the directory. To turn off yes/no prompt for each file transfer, use the prompt command.

I would explicitly run the binary command to ensure your GZip files are downloaded correctly.

[Jaze]

OK, got that sussed via command line but shell script doesn't seem to work.

#! /bin/sh
ftp ftp://user:[e-mail]
binary
prompt
cd /var/log/httpd/archive
mget some-domain-access-log.*.gz
bye

This will login ok and it changes to binary but not because of my command, its like its not receiving the next command as witnessed when I exit via 'bye'

./accesscron: binary: command not found
./accesscron: prompt: command not found
./accesscron: cd: /var/log/httpd/archive: No such file or directory
./accesscron: mget: command not found
./accesscron: bye: command not found

yet if I use these same commands one by one in the terminal myself it does work?

Well mostly, I have this warning:
ftp> mget some-domain-access-log.*.gz
'EPSV': command not understood.
but it still ftps the access log

Any further help?

[Jaze]

Got a reply from web server people - that makes things easier - although I'm still curious as to how to send commands to ftp via shell script. That could be heaps of fun.

I even tried to echo the commands with (for example) `ls`.

But cron on the right day with the MDATE variable will do for now.

[sun818]

Try this in your shell script. Make sure your ftp command ends with <<ENDOFINPUT and the line after your last FTP command is ENDOFINPUT

ftp -n -i <<ENDOFINPUT
open FTPADDRESS
user USERNAME PASSWORD
get FILENAME
close
bye
ENDOFINPUT

[Jaze]

Bingo - thanks!

ftp -n -i <<ENDOFINPUT
open FTPADDRESS
user USERNAME PASSWORD
cd DIRECTORY
binary
prompt
mget FILENAME .*.gz
close
bye
ENDOFINPUT

With Cronnix, once the script had been created, I drag and dropped into the cron entry I had created. Works perfectly :)

thanks sun818

[sun818]

You're welcome. Google is pretty useful ;)