V3 database inflation - format change proposal

Hi,

An interesting issue arose while playing with the V3 format:

A database with ~400 entries takes ~40kbytes to store in the V2 format.
The same data stored in V3 takes up ~62k - over 50% more!

The loading times is also longer by roughly the same amount - about ~40%.

A quick analysis shows that most of the difference is due to the 8
additional wasted bytes in the first block of each field (8 fields per
record).

Therefore, I'm thinking of modifying the field format as follows: Instead of
the first block of a field containing only four length bytes and a type
byte, which left 3 bytes unused in V2 (since Blowfish has a blocksize of 8),
but 11 (!) unused in V3,  the first block will also contain up to 11 bytes
of data.

Graphically: L = Length Byte, T = Type Byte, X = unused byte, D = Data

V2 field:
LLLLTXXX DDDDDDDD...DDDDDDDD...

Current V3 field:
LLLLTXXXXXXXXXXX DDDDDDDDDDDDDDDD...

Proposed V3 field:
LLLLTDDDDDDDDDD DDDDDDDDDDDDDDDD...

(Note that for up to 8 bytes of data, the size of a new V3 field is the same
as V2, and for 9, 10 or 11 bytes it's actually shorter by 8 bytes!)

Any comments on this proposed change?

	Thanks,

		Rony




-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642