Re: Versioning your Model with NoSQL storage

Greg, Rickard, thanks for your answers. So basically we have to strategies:

- Apply a BULK-migration to all versioned data. This is reasonably easy,
but only works for 'small'  datasets and falls into the category of offline
migrations.
- Apply an version-on-the-fly migration which means your infrastructure
must somehow maintain knowledge of previous versions (I assume up to a
certain point) and be able to upconvert old data to the latest version (and
store it in the new format when updated). This works better for large
datasets and/or distributed environments but is more complex and
infrastructure-dependent, is it not?






On Tue, Apr 2, 2013 at 11:40 AM, Rickard Öberg <[email protected]>wrote:

> **
>
>
> On 4/2/13 17:38 , Greg Young wrote:
> >> In one system I wrote the storage was serialized Java objects
> (blindingly
> >> fast read/write was objective), and the changes between 1.0 and 2.0 were
> >> substantial, including aggregate changes. What we did was to serialize
> the
> >> whole database to a single massive XML file, and then use a 2-pass SAX
> XSLT
> >> to transform it into new format, and then chop it up back into the new
> >> serialized format.
> >
> > This works very well on small datasets. If you have a few billion
> > items in say riak (128 node cluster) this doesn't work so well :)
>
> For sure.
>
>
> > There is a secondary problem with this in that this is an "offline"
> migration.
>
> Yup. For online migration the read-fix thing that Qi4j has works better.
>
> /Rickard
>
>  
>