building compatible zip files "by hand" that are encrypted and compressed

Ed Fair <ed_fair-/[email protected]> Thu, 18 Sep 2003 15:12:46 -0700 (PDT)
Newsgroups gmane.comp.archivers.info-zip
Message-ID <[email protected]>
Hello info-zip experts,

I am trying to do something with ZIP files and I'm
wondering if any of you may've done something like
this, or know of someone who has.

I have written some client/server backup software (see
http://www.whlsoftware.com) that uses zip files to
deliver data.  The basic flow of my software is that a
background task watches the client file system for
files needing archival, and, when a file is found, it
sends that file to the server.  The file(s) are not
zipped or preprocessed in any manner before sending,
and the client uses a TCP protocol to send the file(s)
up to the server.   The server receives and catalogs
each file received in this manner, and then saves the
files to disk or tape.  The server does not ZIP or
postprocess the files after receiving them, they are
simply written to disk or tape.

When the end user decides they want to restore one or
more files, they formulate a restore request to the
server, and this is where ZIP archives get involved. 
The server processes the restore request by finding
the requested files and building a ZIP archive; the
zip archive is then made available for the end user to
download, the end user may then extract their files
from this archive using whatever UNZIP utility they
have.

Everything that I've mentioned above works great in
the current version (1.2) of my software.  But there
are  two issues that I need to address: compression
and encryption.  I would like to compress and encrypt
the files before the client sends them to the server. 
Private-key AES is my desired encryption algorithm,
and I don't care too much which compression technique
I use as long as it isn't too CPU intensive.

The refinement I'm after is that client will compress
and encrypt before sending, and the server will
receive and store nothing but compressed and encrypted
files.  Further, when the end user formulates a
restore request, the server will simply insert the
compressed and encrypted files into a ZIP archive. 
The server will still have to create the ZIP file, but
it will not need to perform either compression or
encryption while creating it.

However, since the files are already compressed and
already encrypted, I would like very much for the end
user to be able to use whatever UNZIP utility they
have to extract these files from the archive.  I would
like their UNZIP utility to decrypt and decompress the
files during extraction, prompting for the secret key
when needed.  In other words, I want to make sure that
the files that come out of the Extract are cleartext
versions of the original files.

I must ensure, during client preprocessing, that files
are compressed and encrypted in a manner compatible
with the common UNZIP utilities.  And, I must ensure
that when the server builds a ZIP file, that it sets
the header fields to indicate the compression and
encryption to be used during extraction.

I'm looking for guidance on what libraries are
available, if any, to perform the compression and/or
encryption during client preprocessing.  I'm also
looking for guidance on building ZIP file headers so
that the extract button will do the right thing when
the end user receives the ZIP archive.

Most importantly, I am trying to stay UNZIP-neutral in
all this, so that I don't force the end user into any
camp.

Has anybody else attempted anything like this before?

Any suggestions or insights would be most welcome.

ed



__________________________________
Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com