Re: Versioning your Model with NoSQL storage

On 4/2/13 17:24 , Greg Young wrote:
> These are certainly issues with any storage system. There are a few
> things you can look at.
>
> 1) use weak versioning. Make additive/deleting changes to documents
> don't rename things etc. At that point a simple json serializer should
> help. It will also allow any version to read any version
> 2) use a view system of underlying documents. Transform them into the
> way you want to see them.
>
> There are however some more sinister changes with these models. Namely
> what happens when you have to repartition documents. As an example
> consider Vehicle and Engine that used to be a single aggregate vehicle
> but you realize that engines have lifecycles independent of vehicles
> .... This would require separating the vehicle documents.
>
> These kinds of issues exist in all systems. NoSQL databases are strong
> at handling some (as are SQL databases).

In one system I wrote the storage was serialized Java objects 
(blindingly fast read/write was objective), and the changes between 1.0 
and 2.0 were substantial, including aggregate changes. What we did was 
to serialize the whole database to a single massive XML file, and then 
use a 2-pass SAX XSLT to transform it into new format, and then chop it 
up back into the new serialized format.

It was actually surprisingly pleasant to do it this way, and using XSLT 
for the transformations was fairly intuitive. Because we used SAX for 
parsing it scaled quite well to large databases, the only limitation 
being how much data we needed to keep in memory between first and second 
phase.

Another app I worked on used Qi4j, which stores entities using JSON 
serialization and include app version nr in each object, and has 
subsystem for data migration rules either on the whole database, or 
fix-on-read.

/Rickard



------------------------------------