Re: GZIP
Karl Wettin <[email protected]>
| Newsgroups | gmane.comp.java.prevayler |
|---|---|
| Message-ID | <[email protected]> |
I suppose another but rather hacky solution would be to gzip manually and then open it up using a gzipstream if it match some criteria such as file extension. They all have the same magic numbers... Perhaps the criteria could come from some sort of registry containing meta data such as serializer class and compression for journals and snapshots? It could be as simple as 0000000000000000001.journal.meta.xml and 0000000000000000001.snapshot.meta.xml with a small utility class that would post process files at snapshot time, the previous journals on startup, or what not. I suppose this also would make it simpler for people to switch serializer half way in to the game, if that ever was a problem. karl On Jan 23, 2012, at 7:19 PM, Justin T. Sampson wrote: > It makes conceptual sense, but it turns out to be tricky. The journal needs to ensure that data is synced to disk at specific moments, and compression and encryption tend to have their own internal structure that complicates that. Perhaps the ideal would be invoking compression and encryption at the moment when DurableOutputStream writes a batch to disk, which would indeed require inserting a hook in the middle of the DurableOutputStream code (since it can't simply be a wrapper above or below that level). > > On Wed, Jan 18, 2012 at 11:52 AM, Klaus Wuestefeld <[email protected]> wrote: > Maybe a hook, like the serializer. > > > On Tue, Jan 17, 2012 at 4:27 PM, Karl Wettin <[email protected]> wrote: > > I just managed to compress a GZIPSerializer built journal from 70MB to 5MB using gzip the.journal. > > > > GZIPSerializer deflates each instance with a new header. This means that snapshots are compressed fully while journals is far from it. > > > > I'm starting to think that compression is something that should be implemented in DurableOutputStream. > > > > What do you say? > > > > > > > > karl > > > ------------------------------------------------------------------------------ > Try before you buy = See our experts in action! > The most comprehensive online learning library for Microsoft developers > is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, > Metro Style Apps, more. Free future releases when you subscribe now! > http://p.sf.net/sfu/learndevnow-dev2_______________________________________________ > To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion > _______________________________________________ > "Databases in Memoriam" -- http://www.prevayler.org ------------------------------------------------------------------------------ Try before you buy = See our experts in action! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-dev2 _______________________________________________ To unsubscribe go to the end of this page: http://lists.sourceforge.net/lists/listinfo/prevayler-discussion _______________________________________________ "Databases in Memoriam" -- http://www.prevayler.org