Re: Savepoint

Hello Anthony,

I think you are mostly right. The only reason I made savepoint was to
have a more fine grained behaviour, allowing th "clone" only the records
that where dirty... But I really should have first made some
measurements about the efficiency of the current clone implementation,
tried alternative implementations, and then maybe try the savepoint
approach...

So, to sum, the goal I have (being able to reuse the dataset in case of
an error), would be achieved by clone().

Savepoint is more intrusive, but as a conterpart, is also more fine
grained (only duplicate dat that has changed).

What I will do is create some benchmarks, and then compare the clone and
savepoint approaches.

Then I'll see if I can improve the clone approach...

I'll try to do this asap so we can decide if the savepoint is to be
removed altogether...

I'll kepp you informed.

Thanks again for your input,

Franck


Le vendredi 07 août 2009 à 12:28 +1000, [email protected] a écrit :
>   
> Hello Franck,
> 
> OK, I now see what you mean about clone() and serializable. 
> 
> But I do not see why clone is inherently less inefficient than what
> you have done. The only extra objects that need to be
> SRecrordInstances themselves. There is much more work in copying the
> array lists, so the overhead is negligible. Certainly a properly
> implemented clone approach makes much fewer changes to our data
> structures. And I am pretty sure that save points and clone should not
> both be supported.
> 
> But what if the SRecordInstances have business logic. You set a save
> point, updates some fields (and thus business rules), and then do a
> rollback. What happens? Should rolling back to a save point fire
> business rules? (Imagine, for example, a UI driven by
> onValidateRecord, say.)
> 
> Hard questions.
> 
> My current feeling is that we should implement clone efficiently
> (without serialization) and then leave it to the user. Maybe can only
> clone if not attached. This adds minimal complexity to the main line
> code, and gives you what you want.
> 
> The problem is that Java's clone is a mess. (If we have a clone()
> method, should we implement Clonable?) But it does allow the user to
> extend clone() for SRecordInstance and thus control the process.
> 
> I have not worked with clone before. But I think that if
> SDataSet.clone and more importantly SRecordInstance.clone call
> super.clone, then you will have a shallow copy. You then just need to
> explicitly clone the mutable objects, such as Arrays. Try not to clone
> immutable objects -- String does not implement Clone. Maybe just never
> clone the values within the values array at all. 
> 
> A user might want to implement Employee.clone, say, but not for your
> case.
> 
> If we take the clone approach, then all references to Savepoints are
> removed from the API. It is just a technique to clone before attaching
> and running. All changes to the data structures also go.
> 
> As a third alternative maybe not optimize clone at all. Just stick to
> the serialization approach. Have we done any measurements to see if it
> is actually inefficient? One problem with introducing clones is that
> if the user was doing clever things with business rules then they
> would have to deal with the clone case separately from the serialize
> case. Really bad Java design that the two are not unified in any way.
> But that is the way it is. (The main inefficiency of Clone would be
> re-creation of the String objects whose references could be otherwise
> shared.)
> 
> Can you see any problem with the clone approach? It seems cleaner to
> me.
> 
> Regards,
> 
> Anthony
> 
> At 07:07 PM 5/08/2009, you wrote:
> > 
> >
> >Le mercredi 05 août 2009 à 14:05 +1000, <mailto:berglas%
> 40SpreadsheetDetective.com>[email protected]
> >a écrit :
> >
> >> 
> >> Flush failing is not the same as a rollback.
> >
> >Right, but most of the time, when flush fails, I will rollback, as I
> >don't have enougth information to recover from the situation.
> >> But I do see your point. Continuing on would be very hard, some
> dirty
> >> bits cleared, others not etc.
> >Yep.
> >> 
> >> One of the common cases would be a broken optimistic lock.
> >
> >Yes, this is a typical case. But also any broken constraint or
> >SValidation failing.
> >When the user submits some data, there are many reasons why the data
> >might not fit into the database (although, of course, we make our
> best
> >to assist the user and help him/her avoid submitting stupid things).
> >
> >> I suppose we could purge the broken record and re read it,
> hopefully
> >> keeping other records intact. 
> >True in the case of optimistic lock, but the broken record might also
> be
> >a new record, in which case we cannot re-read it...
> >> 
> >> In removeRecord 
> >> if ( ! hasSavepoint()) rinst.destroy();
> >> is a bit dubious. destroy() really takes on two meanings.
> >
> >Previous code was 'rinst.destroy()'. In my try to keep a possibility
> >restore the dataSet, I don't destroy the record, as a reference is
> kept
> >to it in spRecords and spDirtyRecords.
> >So removeRecord means 'remove it from the dataSet in its current
> state,
> >and cleanup if possible, but cleanup is not the main goal of
> >removeRecord(), it is the goal of SDataSet.destroy()'.
> >
> >> And what do we do about references to removed Record?
> >> 
> >Well, anyway SRecordInstance.destroy() does not do anything about
> >references. So I think my modification has no impact on this
> matter...
> >
> >But the point remain valid, what happens to references to a
> >removedRecord? They probably should be set to null, which is not done
> >now. Maybe the first thing to do would be create a testcase...
> >I'll try to do this.
> >
> >> Would it be better to just clone the SRecordInstances and indeed
> the
> >> whole SDataSet? Certainly fewer changes to the data structures.
> More
> >> flexible, a little less controllable. What was your reasoning?
> >> 
> >This is was is done in SDataSetclone(). I think it works, but is
> costly.
> >My goal was the reduce the amount of data the is copied, ie only make
> a
> >copy of the data in SDataSet, and a copy of the data in _dirty_
> records
> >only. There are situation where the number of dirty records in a
> dataSet
> >is much lower than the total number of records...
> >
> >> And can we think of an elegant way to handle broken optimistic
> locks?
> >> (Often it will just need to be thrown back to the user.)
> >> 
> >Before sorm3 and datSets, all these problems were easier to handle:
> >simply detach the records, send them back to UI, re-attach them
> later.
> >But the point is we had no guarantee of thing being coherent!! The
> >SDataSet introduces a level of coherence, where all records live
> >together, or die together... Hence the problem I try to solve...
> >
> >> (I'm pretty sure that all this goes beyond Hibernate when sessions
> are
> >> saved.)
> >> 
> >No idea what Hibernate does... I've not looked at it for years, since
> I
> >know Simpleorm :) . I know I should :(
> >
> >> Anthony
> >> 
> >> At 05:18 AM 5/08/2009, you wrote:
> >> > 
> >> >
> >> >Hello Anthony,
> >> >
> >> >it is indeed something we need...
> >> >
> >> >The use case is the following:
> >> >
> >> >1) the user creates a dataSet and some records in the UI layer.
> >> >
> >> >2) the dataSet is sent to the data access layer, where a session
> is
> >> >created and the dataSet is bound to it.
> >> >
> >> >3) the flush fails for whatever reason, throwing an exception.
> >> Session
> >> >rollbacks. Thanks to savepoint, the dataset is NOT destroyed, it
> is
> >> >restored in the state it was when it came from the UI layer.
> >> >
> >> >4) error are mapped to the fields, and the dataSet is sent back to
> >> the
> >> >user to allow corrections to be made
> >> >
> >> >Right now (without savepoint), the dataSet is destroyed, and the
> user
> >> >input is lost, with a sorry error message...
> >> >
> >> >Well, the truth is we could hold the user input in another data
> >> >structure in the UI, but our application is built around dataSet.
> So
> >> the
> >> >UI is build from the dataSet, either when reading from the
> database,
> >> or
> >> >when creating data...
> >> >
> >> >We avoid the "sorry your input is lost" scenario by testing as
> much
> >> >failure case as we can before sending the dataSet to the data
> access
> >> >layer: we manually test the foreign keys for each record, we
> manually
> >> >test the unique constraints, etc... this is costly and not very
> >> >efficient. And sometimes it fails.
> >> >
> >> >So having the ability to "rollback" a dataSet instead of
> destroying
> >> it
> >> >will really change our life...
> >> >
> >> >Whether my code does the job remains to be seen :)
> >> >
> >> >Does it make sense to you ?
> >> >
> >> >Regards,
> >> >Franck
> >> >
> >> >Le mardi 04 août 2009 à 20:04 +1000, <mailto:berglas%
> >> 40SpreadsheetDetective.com><mailto:berglas%
> 40SpreadsheetDetective.com>[email protected] a
> >> >écrit :
> >> >> 
> >> >> Hello Franck,
> >> >> 
> >> >> Yipes. Is this something that you really needed? 
> >> >> 
> >> >> I'm guessing as a way to recover from broken optimistic locking.
> >> But
> >> >> that would not work -- there would be no way to correct the
> >> situation.
> >> >> 
> >> >> This sort of thing can lead to many issues, and needs thought.
> What
> >> is
> >> >> your use case?
> >> >> 
> >> >> We should probably take care that if a flush() fails, then
> >> unflushed
> >> >> records remain in a state that they can be flushed again. I have
> >> not
> >> >> looked, but should probably be the case already.
> >> >> 
> >> >> It should cost the same as clone(), as that it what it does. And
> >> maybe
> >> >> clone would be more useful?
> >> >> 
> >> >> I see you have a test case, have not had a chance to look at it
> >> >> carefully.
> >> >> 
> >> >> But what is your use case?
> >> >> 
> >> >> Let us put Map to bed with a test case, and then have a review
> of
> >> >> aggregate, before adding too many more features.
> >> >> 
> >> >> Regards,
> >> >> 
> >> >> Anthony
> >> >> 
> >> >> At 02:23 AM 3/08/2009, you wrote:
> >> >> > 
> >> >> >
> >> >> >Hi Anthony,
> >> >> >
> >> >> >I couldn't help but commit a new feature whose intent is to
> allow
> >> to
> >> >> >recover a dataset after a rollback, without the cost of the
> >> clone()
> >> >> >solution.
> >> >> >
> >> >> >If not used, the feature has no impact on the way simpleorm
> works,
> >> so
> >> >> I
> >> >> >thought it was quite harmful...
> >> >> >
> >> >> >I called it savepoint, and the usage is:
> >> >> >
> >> >> >ses.begin(ds.savepoint());
> >> >> >
> >> >> >then, if 'ds.hasSavepoint()', you can:
> >> >> >
> >> >> >ses.rollbackAndDetachDataset();
> >> >> >
> >> >> >The tradeoff is that ds.savepoint() is quite costly, especially
> >> for
> >> >> big
> >> >> >dataSets... (but should be less costly than clone()).
> >> >> >
> >> >> >Of course, comments are welcome !
> >> >> >
> >> >> >Franck
> >> >> >
> >> >> >
> >> >> 
> >> >> Spreadsheet Detective,
> >> >> Southern Cross Software Queensland Pty Limited
> >> >> 54 Gerler Street
> >> >> Bardon, Queensland 4065, Australia.
> >> >> 
> >> >> Email: <mailto:berglas%
> >> 40spreadsheetdetective.com><mailto:berglas%
> 40spreadsheetdetective.com>[email protected]
> >> >> www.SpreadsheetDetective.com
> >> >> Ph: +61 427 830248 (Australian Eastern Standard Time)
> >> >> 
> >> >> "If the model seems correct only because the numbers look
> right, 
> >> >> then why build the model in the first place?"
> >> >> 
> >> >> 
> >> >> 
> >> >> 
> >> >
> >> >
> >> 
> >> Spreadsheet Detective,
> >> Southern Cross Software Queensland Pty Limited
> >> 54 Gerler Street
> >> Bardon, Queensland 4065, Australia.
> >> 
> >> Email: <mailto:berglas%
> 40spreadsheetdetective.com>[email protected]
> >> www.SpreadsheetDetective.com
> >> Ph: +61 427 830248 (Australian Eastern Standard Time)
> >> 
> >> "If the model seems correct only because the numbers look right, 
> >> then why build the model in the first place?"
> >> 
> >> 
> >> 
> >> 
> >> 
> >
> >
> 
> Dr Anthony Berglas, [email protected] Mobile: +61 4 4838 8874
> Just because it is possible to push twigs along the ground with ones
> nose
> does not necessarily mean that is the best way to collect firewood.
> 
> 
> 
> 
> 





------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/SimpleORM/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/SimpleORM/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:[email protected] 
    mailto:[email protected]

<*> To unsubscribe from this group, send an email to:
    [email protected]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/