Re: Shipping a build

Hi,

there is a point I don't get... there are big differences between
SRecordGeneric and SRecordInstance, that make me think it would be hard
to unify them:

SRecordInstance can be persisted,
SRecordInstance have a primary key and are guaranteed to be unique
within a session/dataset,
SRecordInstance can have optimistic values, etc.

On the other hand, SRecordGeneric are really read-only,
SRecordGeneric cannot be updated,
SRecordGeneric will be unique only per query result, not on a
transaction/dataset basis (they have no primary key).

So I understand how it would be interesting to be able to add arbitrary
read-only fields to SRecordInstance as in you example (and make it
implement Map or even backed by a Map), but I don't get how we could
pretend SRecordInstance and SRecordGeneric are the same beast...

What do you think of it ?

Franck


Le mercredi 29 juillet 2009 à 13:09 +1000, [email protected] a écrit :
>   
> Hello Franck,
> 
> Good to hear that you are back on this. Producing a new release is way
> over due. I have also been busy.
> 
> The reasoning is that aggregates is only one thing that it can do. The
> thing that is special about it is that it returns SRecordGenerics
> rather than SRecordInstances, with select list specified. It can do
> much more than simple aggregates, even though that was the initial
> motivation.
> 
> While in this space I would love to unify SRecordGeneric,
> SRecordInstance and Maps. It is a messy at the moment. A lazy
> incremental approach is as follows.
> 
> WRONG WAY?
> 
> SRecordGeneric becomes an abstract class with stubs for
> getDouble(String) etc.
> 
> SRecordInstance and a new SRecordMap both extend SRecordGeneric.
> 
> SQuerySelect returns lists of SRecordMap. SRecordMap is not a public
> class, just package local.
> 
> SRecordGeneric implements Map. For SRecordInstance that means
> implementing the map methods to call Instance methods. This makes it
> much more compatible with JSP, JSF etc.
> 
> The copied code in SRecordMap from SRecordInstance that does type
> conversion is pushed into SRecordGeneric. (I really hate copied code.
> It invariably drifts apart. At the very least create common static
> methods.)
> 
> Ideally, and probably later, SDataSets can contain SRecordGenerics
> rather than SRecordInstances. 
> 
> Also, SRecordInstances should implement a Map of fields rather than an
> Array of fields. Then SRecordInstances can have additional, undeclared
> fields, and so SRecordMap disappears. I have done some experiments and
> the overhead is negligible even given Map's unfortunate implementation
> (an extra object for every key/value pair -- exactly what I was
> avoiding with not having SFieldInstance).
> 
> CLEANER ALTERNATIVE
> 
> The above is really a bit back to front. I wonder how hard it would
> really be to take the array out of SRecordInstance and just replace it
> with a Map that it implements, indexed by field name. As stated
> before, the overhead is minimal. Then your SQuerySelect could just
> return SRecordInstances, and there would be no need for
> SRecordGeneric. SQuerySelect should be unified with SQuery, so that ad
> hoc columns can be combined with known fields. 
> 
> For example, Select Department Name, Manager, AVG(employee Sallary) as
> Department record instances. Could then do dept.getString(MANAGER),
> dept.getString("avgSal"). (Given that "avgSal" is an ad hoc field not
> declared for Department.
> 
> Much neater. And demonstrates the advantage of not using rigid POJOs.
> (Will also behave better if there are deserialization issues.)
> 
> The more I think of this the more I do not like adding SRecordGeneric.
> It forks the concept of Record.
> 
> The Alternative is probably less work in the medium turn. Just making
> the field index a map should be fairly trivial -- we just remove
> SFieldMeta.index as it is no longer needed and replace
> 
> public Object getRawArrayValue(SFieldMeta fmeta) {
> return fieldValues[fmeta.index];
> }
> 
> with
> 
> public Object getRawArrayValue(SFieldMeta fmeta) {
> return this.get(fmeta.fieldName);
> }
> 
> I'll do the latter if you will remove SRecordGeneric and unify the
> SQueries. 
> 
> (SQueryAggregate just becomes additional methods to SQuery. Or maybe
> delegated from SQuery as SQuery is getting too large.)
> 
> What do you think?
> 
> Regards,
> 
> Anthony
> 
> At 02:44 AM 29/07/2009, you wrote:
> > 
> >
> >Hi Anthony,
> >
> >it's been a long time since I said I would rename SAggregateQuery to
> >SQuerySelect, as soon as... well. I'm ready now :)
> >
> >I just come back to make sure I understand you:
> >
> >We want to rename SAggregateQuery to SQuerySelect to make the name
> more
> >generic, right ?
> >
> >But right now, SAggregateQuery is really only capable of returning
> >aggregates, that is SRecordGeneric whose fields are sums, averages,
> etc.
> >The specific methods of SAggregateQuery require that you do
> aggregates
> >and won't allow you to query arbitrary fields out of joined tables
> for
> >example.
> >
> >SO, do you want to rename SAggregateQuery in order to make it more
> >generic afterwards, or do you mean SAggreagateQuery and SQuery should
> >share a common interface or a common base (abstract ?) class that
> would
> >be called SQuerySelect ?
> >
> >Regards,
> >
> >Franck
> >
> >Le dimanche 24 mai 2009 Ã 17:52 +1000, <mailto:berglas%
> 40SpreadsheetDetective.com>[email protected]
> >a écrit :
> >> 
> >> 
> >> Sounds good. I'll have some time over the next few weeks.
> >> 
> >> Anthony
> >> 
> >> At 08:19 PM 18/05/2009, you wrote:
> >> 
> >> >Hello,
> >> >
> >> >I have a few other commits waiting here that I would like to go in
> >> the
> >> >build.
> >> >
> >> >Mainly, it allows to do:
> >> >
> >> >aggQuery.sum("some arbitrary sql here")
> >> >
> >> >(or groupBy, avg, max, min, count)
> >> >
> >> >For example sum("case where myfield = "TRUE" then 1 when
> >> myfield="FALSE"
> >> >then 0 end")
> >> >or
> >> >sum("(exract(epoch from myEndTimestamp) - extract(epoch from
> >> >myBeginTimestamp")) / 3600") as durationindays
> >> >
> >> >I also added a hasTable(alias) method to SQuery to ease building
> of
> >> >conditionnal quieries.
> >> >
> >> >I'll commit it to let you have a look if you want.
> >> >
> >> >Then I'll rename SAggregateQuery to SQuerySelect...
> >> >
> >> >Regards,
> >> >Franck
> >> >
> >> >Le lundi 18 mai 2009 Ã 14:02 +1000, <mailto:anthony%
> >> 40berglas.org><mailto:anthony%40berglas.org>[email protected] a
> écrit :
> >> >> 
> >> >> 
> >> >> Hello Franck,
> >> >> 
> >> >> I would like to package up the current subversion state fairly
> soon
> >> >> and make it the next build. There are a few tweak I would add.
> >> >> 
> >> >> Could you have a bit of a think about what needs to be cleaned
> up
> >> etc.
> >> >> for this to happen. Rename SQueryAggregate to something like
> >> >> SQuerySelect is one.
> >> >> 
> >> >> I'd prefer that you made changes related to your application so
> >> that
> >> >> you do not break compatibility.
> >> >> 
> >> >> Regards,
> >> >> 
> >> >> Anthony
> >> >> 
> >> >> >Hello Franck,
> >> >> >
> >> >> >I see your point about SQueryGeneric. How about SQuerySelect,
> >> because
> >> >> it lets you explicitly specify a select list? 
> >> >> >
> >> >> >I realize that you are busy but it would be good to at least do
> >> just
> >> >> the rename plus comment as soon as possible. (I won't do it
> because
> >> it
> >> >> will break your code.)
> >> >> >
> >> >> >I am a little concerned about getting too clever in SDriver
> about
> >> >> portability. Supporting this type of thing for multiple
> databases
> >> is
> >> >> very hard. I think that I would prefer that you just do this at
> the
> >> >> application layer. You just have little methods that provide the
> >> right
> >> >> query for different DBs. Could just return strings, or possibly
> >> >> SQuerySelect (given an SQuerySelect parameter). I prefer
> strings,
> >> it
> >> >> is simpler.
> >> >> >
> >> >> >Regards,
> >> >> >
> >> >> >Anthony
> >> >> >
> >> >> >At 04:49 PM 19/04/2009, you wrote:
> >> >> >>Hello Anthony,
> >> >> >>
> >> >> >>I've not be changing anything recently, except adding a
> >> rawInnerJoin
> >> >> >>method to SQuery, that truly adds an arbitrary table to a
> query
> >> (a
> >> >> table
> >> >> >>that has no explicit SFieldReference to the query).
> >> >> >>
> >> >> >>But I have also found I would need rawSelect(), so it is
> >> definitely
> >> >> a
> >> >> >>good idea :-)
> >> >> >>
> >> >> >>The point is we are also releasing a new version of our
> flgship
> >> >> product
> >> >> >>by the end of the month, so I'll be quite busy right until
> >> then...
> >> >> >>
> >> >> >>Just a few more comments:
> >> >> >>
> >> >> >>> I think that specifically SQueryAggregate and friends should
> be
> >> >> renamed into something like SQueryGeneric. That is because the
> >> thing
> >> >> that is special about it is that it returns SRecordGenerics
> rather
> >> >> than SRecordInstances. One use of SQueryGeneric is to do
> >> aggregates,
> >> >> but it is more general than that.
> >> >> >>
> >> >> >>Yes, but then, when SRecordInstance will extend
> SRecordGeneric,
> >> >> SQuery
> >> >> >>will also return SRecordGeneric... So, are we going to have an
> >> >> abstract
> >> >> >>SQueryGeneric, with SQuery and SAggregateQuery extending it,
> or
> >> do
> >> >> you
> >> >> >>think both should be merged in the long term ? (right now I
> >> wouldn't
> >> >> say
> >> >> >>so, but I'm not sure)
> >> >> >>> 
> >> >> >>> Thus we should also add the easy 
> >> >> >>> .rawSelect(String rawSqlExpression, Object[]... parameters) 
> >> >> >>> which adds an arbitrary an arbitrary expression,
> >> plus .rawClause
> >> >> for completeness.
> >> >> >>
> >> >> >>Yes, that would be interesting...
> >> >> >>Another thing I found I'm missing is doing arbitray
> calculation
> >> and
> >> >> then
> >> >> >>doing an aggregate on it, but rawSelect wouldn't solve my
> problem
> >> >> >>altogether, as calculation might include database functions
> that
> >> are
> >> >> not
> >> >> >>standard.
> >> >> >>Typical example is I want to make a sum of durations for my
> >> records,
> >> >> >>grouped by whatever. Duration is defined by the difference
> >> between
> >> >> two
> >> >> >>timestamps... and this is something that in not standard in
> sql.
> >> So
> >> >> >>having portable code is not easy.
> >> >> >>
> >> >> >>eg. select sum( extract(epoch from endts) - extract(epoch from
> >> >> begints))
> >> >> >>will work for postgresql, but not for oracle etc.
> >> >> >>
> >> >> >>So another idea I had was to be able to augment SDriver (and
> >> >> >>SQueryGeneric) with specific functions (like duration(ts1,
> ts1))
> >> >> that
> >> >> >>could be used in queries to allow to generate sql that will
> work
> >> in
> >> >> each
> >> >> >>database... I didn't do anything for now, as I'm not sure
> about
> >> the
> >> >> >>right way to do this (just add the functions I need, or think
> >> about
> >> >> a
> >> >> >>more clever pluggable functions system, ...)
> >> >> >>
> >> >> >>> 
> >> >> >>> The simple unification of SRecordInstance with
> SRecordGeneric
> >> >> would complete the process, but that can be done later. I might
> >> make
> >> >> time do it myself because it will not affect your code. (It is
> >> easy,
> >> >> just have SRecordInstance implement map, with 
> >> >> >>> 
> >> >> >>Fine.
> >> >> >>As I said, I won't have much time for the next two or hree
> weeks,
> >> >> but it
> >> >> >>should be better after that.
> >> >> >>
> >> >> >>Best regards,
> >> >> >>Franck
> >> >> >
> >> >> >Dr Anthony Berglas, <mailto:anthony%
> 40berglas.org><mailto:anthony%
> >> 40berglas.org><mailto:anthony%40berglas.org>[email protected]
> >> >> Mobile: +61 4 4838 8874
> >> >> >Just because it is possible to push twigs along the ground with
> >> ones
> >> >> nose
> >> >> >does not necessarily mean that is the best way to collect
> >> firewood.
> >> >> >
> >> >> >
> >> >> 
> >> >> Dr Anthony Berglas, <mailto:anthony%
> >> 40berglas.org><mailto:anthony%40berglas.org>[email protected]
> Mobile: +61 4 4838 8874
> >> >> Just because it is possible to push twigs along the ground with
> >> ones
> >> >> nose
> >> >> does not necessarily mean that is the best way to collect
> firewood.
> >> >> 
> >> >> 
> >> >> 
> >> >> 
> >> >> 
> >> >
> >> >
> >> 
> >> Spreadsheet Detective,
> >> Southern Cross Software Queensland Pty Limited
> >> 54 Gerler Street
> >> Bardon, Queensland 4065, Australia.
> >> 
> >> Email: <mailto:berglas%
> 40spreadsheetdetective.com>[email protected]
> >> www.SpreadsheetDetective.com
> >> Ph: +61 427 830248 (Australian Eastern Standard Time)
> >> 
> >> "If the model seems correct only because the numbers look right, 
> >> then why build the model in the first place?"
> >> 
> >> 
> >> 
> >> 
> >> 
> >
> >
> 
> Dr Anthony Berglas, [email protected] Mobile: +61 4 4838 8874
> Just because it is possible to push twigs along the ground with ones
> nose
> does not necessarily mean that is the best way to collect firewood.
> 
> 
> 
> 
> 





------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/SimpleORM/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/SimpleORM/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:[email protected] 
    mailto:[email protected]

<*> To unsubscribe from this group, send an email to:
    [email protected]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/