Re: Need to ship SimpleOrm, Benchmarks

Hello Franck,

Looks like we need another top to bottom review of the findOrCreate new row business.  This sort of thing needs to be right.  I will be busy for the next few days, but want to sort this out and ship.

I do not trust my own bench marking -- the numbers are not consistent.  (They never are with Java, JITs etc.)  Needs more work.  That's why I was hoping to benchmark with your actual application.

I'd be more interested to know how long it takes to clone 1000 datasets with 10 records each.  I suspect that very large numbers of records with just a few dirty is an odd case.

And we need to test clone().

As to Dates and BigDecimal, should we clone them?  They are mutable in theory, but we only ever create new instances of them (I hope).  I think the shallow copy is OK, with a warning in the docs.  If people start mutating these objects after doing the set* then we will have many failures indeed.  (e.g. optimistic values would change without our changing them!)

Forking a version is dangerous, I want to make sure that there is only one head.  Are you actually using any of the savePoint code?  Are you actually using the clone code?  

I'd like to write a clone method that roughly clones record set's deeply.  Then measure it.  Then, if there are no surprises, kill the save point code.  In fact, I am very confident that there will be no surprises because the cost of creating the record in the first place will be much bigger than cloning it because the leaf data types do not need to be copied.

But I would really like to clean it all up very soon and ship.

Regards,

Anthony


At 06:03 AM 19/08/2009, you wrote:
>  
>
>Hello Anthony,
>
>I didn't have much time the last days, but I did some tests derived from
>examples/benchmarks
>
>Doing the test I also noticed there is a bug in
>SDataSet.findOrCreate() : if the record is not found and is created, it
>is not marked as new row and will fail when flushing with an
>OptimisticLockException (as all optimistic values are null).
>There is a comment saying:
>// We leave it not new and not dirty. That is set by the SSession.
>but it is definitely not the case if the record is created while the
>dataset is not attached...
>
>I didn't fix anything yet, I'm not sure what to do...
>
>Regarding the results of the tests, on my machine, cloning a 10000
>records dataset takes consistently about 1 sec, just for cloning.
>
>------ simpleormSavepointInsert 288 (291) (microsecond/row)
>------ simpleormCloneInsert 402 (291) (microsecond/row)
>(cloning time 1032 ms)
>------ simpleormInsert 258 (291) (microsecond/row)
>
>So the overhead for cloning is important... (and in the example, all
>records are dirty, so savepoint is not advantaged).
>But on the other hand savepoint is not mature enougth to ship. For
>example I realized that Arrays.copyOf does a flat copy of the array. So
>savepoint works for fields based on immutable objects (Strings, raw
>types, ...) but not for SFieldBigDecimal for example.
>So I would have to make a deeper copy of the arrays in SRecordInstance,
>which will take more time...
>
>What I propose is to remove savepoint for now.
>
>I'll keep a patch on a local copy the test and improve. If one day I
>think it could work, I'll propose it again, but probably as an external
>plugin, relying on hooks in simpelorm core, as it was done for
>SConnectionEJB. 
>
>Does it seem ok for you ?
>
>Regards,
>Franck
>
>Le mardi 18 août 2009 à  14:21 +1000, <mailto:anthony%40berglas.org>[email protected] a écrit :
>> 
>> Hello Frank,
>> 
>> I have updated the white paper, added Transient Query section. Will
>> add a bit more soon.
>> 
>> I have done some rough benchmarks on the serializing clone. Wildly
>> varying results, but about 30ms for a data set with 1000 records, 10ms
>> if about 10 records. So not terrible, but not great.
>> 
>> Made minor changes to genericizations. Be sure to update.
>> 
>> I really want to ship another version while I have a little time. I
>> would like to benchmark clone() on your actual application. And then
>> remove save points, and/or implement clone efficiently. Cannot ship
>> with save points unless we intend to keep them permanently.
>> 
>> I want to do some JSP tests with the maps.
>> 
>> Regards,
>> 
>> Anthony
>> 
>> Dr Anthony Berglas, <mailto:anthony%40berglas.org>[email protected] Mobile: +61 4 4838 8874
>> Just because it is possible to push twigs along the ground with ones
>> nose
>> does not necessarily mean that is the best way to collect firewood.
>> 
>> 
>> 
>> 
>> 
>
>

Dr Anthony Berglas, [email protected]       Mobile: +61 4 4838 8874
Just because it is possible to push twigs along the ground with ones nose
does not necessarily mean that is the best way to collect firewood.



------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/SimpleORM/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/SimpleORM/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:[email protected] 
    mailto:[email protected]

<*> To unsubscribe from this group, send an email to:
    [email protected]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/