How can I automate downloading of my apache web access logs?

Anonymous Blackbaud User

Your apache weblogs are available for download, but they are not available via FTP. What you need to do to automate these downloads is to use a command line HTTP client to perform the download, instead of using a browser.

We recommend the Perl-based lib-www-perl (LWP) toolkit. If you are using e.g. a Red Hat Linux log server, Perl and LWP will already be installed, and there is a convenient command line tool lwp-request provided as part of the kit, e.g.

If you're using NT ... well, you should use Unix if you want automation but seriously, the install of Perl is not difficult (there are SETUP.EXE's available), and is worth the effort for other purposes too. Once you have Perl, there is an automated tool called CPAN which downloads and installs add-on modules - you want to get "Bundle::LWP"

Following is a shell script snippet that could be used to automate these downloads from a Unix command line.

-

---

short=yourConvioShortname #put your actual convio shortname here

url=yourConvioDomain #put your actual convio Domain name here

day=`date --date="yesterday" +%Y%m%d`

filename=$short.access_log.$day.raw

lwp-request -H 'Accept-Encoding: gzip' -C 'linkASSWORD' http://$domain/web_usage_logs/raw/$file >$file.gz

-

---