Re: coding an Application in perl

On 02/23/2016 03:48 PM, Chapman Flack wrote:
> Well, I have the following several ideas for a sequence of
> increasingly specialized apps.
> 
> I like the idea that the apps are Perl classes, and I think they
> could make even more extensive use of object design and inheritance,
> so that a new app could be written in very little code, inheriting
> from a base that does a lot of the work.

I am nearly at the point of pushing this stuff to a github fork
for possible eventual consideration. It has been running for me
in actual backups for the better part of a year (down to a month
or two for the newest scripts), so there is experience behind it.
It adds:

  Amanda::Application::Abstract - a subclass of Amanda::Application
    that does a lot more of the common work every application
    needs, so an application developed as a subclass of
    Amanda::Application::Abstract can be a lot shorter and clearer
    than one that only subclasses Amanda::Application directly.

  Amanda::Application::AbstractScript - the same thing, only for
    scripts. On second thought, Amanda::Script::Abstract might be
    a better name; it won't be hard to change if that's preferred.

Applications:

  amooraw - pure proof-of-concept, same function as amraw but in
    more OO style based on Amanda::Application::Abstract to show
    how much simpler it gets.

  amgrowingfile - can back up (with incremental levels) a single
    large file that is known to grow only by appending (an OS
    may have an 'append-only' file attribute that can be set
    to enforce that). Such a file might be produced by an audit
    or logging system. If it gets rewritten/rotated/truncated,
    a new level 0 should be forced.

  amgrowingzip - analogous to amgrowingfile, but for a single
    large ZIP archive that only changes by having new members
    added at the end. Such a file can efficiently house a
    historical collection of smaller files from an audit or
    logging system, for example. The tool used to add members
    has to be well-chosen, as some zip tools always rewrite the
    whole archive with any change. Any time the whole archive is
    rewritten, a level 0 should be forced. The tool adding members
    should honor advisory locking if it could run at the same time
    as Amanda.

  amopaquetree - tailored to a directory tree whose internal
    structure and file naming are obscure to humans (storage areas
    for databases, version control systems, etc.) and that may
    also contain large files that are only changed in small regions
    (which would cause a traditional application like tar to
    include the whole changed file in an incremental level).
    For DLEs where this is appropriate, amopaquetree trades off
    larger local storage requirement on the client, increased
    computation on the client, and no ability to selectively restore
    individual files, in favor of minimized network bandwidth and
    Amanda holdingdisk/tape space where increment sizes only reflect
    regions of files actually changed.

Scripts:

  amsvnmakehotcopy - uses "svnadmin hotcopy" to get a consistent
    point-in-time copy of the files backing a Subversion repository,
    which can then be backed up by amopaquetree or a more traditional
    application.

  amlvmsnapshot - takes a snapshot of an LVM volume and mounts it
    readonly at a different mountpoint where an Amanda application
    can back it up.

  amlibvirtfsfreeze - on a system that is hosting guest virtual
    machines, uses libvirt (which abstracts operations over several
    different underlying virtualization technologies) to tell the
    OS running in a guest VM to sync and freeze a filesystem, so
    that guest's filesystem image will not be dirty when included
    in the host's backup. Obviously, write operations in the guest
    can block during the freeze, so this is best combined with
    amlvmsnapshot on the host, so the guest can be thawed immediately
    after the host snapshot has been taken.

  am389bak - uses "db2bak" on a 389 Directory Server instance to
    get a consistent point-in-time copy of its backing database
    files, which can then be backed up by amopaquetree or a more
    traditional application.

This is not fully polished yet. The modules have POD and the
applications/scripts have manpages. Nothing has installcheck
scripts yet. Error checking is minimal so far, and there are
surely places that would still die() with a short message
instead of creating a proper Amanda protocol response.

But things are probably at the stage where what's left to be
polished would be readily spotted by getting more eyes on the code.
(It might also be a sound testing practice for installcheck scripts
to be written by someone other than me, though I understand if
no one has that kind of time.)

I notice that the following "support" items are now checked
in client-util.c, beyond what there were when I first wrote
Amanda::Application::Abstract:

 MESSAGE-SELFCHECK-JSON
 MESSAGE-ESTIMATE-JSON
 MESSAGE-BACKUP-JSON
 MESSAGE-RESTORE-JSON
 MESSAGE-VALIDATE-JSON
 MESSAGE-INDEX-JSON

 DISCOVER

These aren't explained yet in the Application-API document; the
first six I suspect have to do with Amanda::Message (which I think
I understand, but none of my code uses yet). Do I need to look at
any of that first, or is it ok to just publish the code to github
as it is now, so everyone can chime in on what further work it needs?

Thanks,
-Chap