Re: DBTransactions and DBConnections

This is a multi-part message in MIME format...

------------=_1112202052-14665-163
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Hi,

Thanks a lot for your comments. We had a lot of discussion in our team when
we implemented autowrite and autosave options.

First you are right that you can get 'wrong' result if you doesn't use
autowrite option and has some modifications in transaction - specially
creating new DOs, without calling write method. We decided to implement
following scenario:
if you use autosave and autowrite and transaction cache in your application,
DODS should guarantee to you that you always get right results. That's a way
how application should use DODS transactions (nice DO aggregation).

If for some reasons, you can't use autowrite and autosave options,
application (developer) must provide that write() method will be called
after some modifications in transaction which can cause wrong query results
in the same transaction.

We can discuss a lot about this 'feature' is it a right way or not. We just
want to have clean situation. If autowrite is false, we don't want that DODS
automatically calls write method without application control.

By the way, DODS has a modular structure and you can change your templates
and use 'custom' templates or you can easily write new Transaction
implementation and use it in your application (setting in conf file which
implementation will be used)

Thanks again for your suggestions. These are always very useful for us.

Regards,
Sinisa


----- Original Message -----
From: "Michael Strapp" <[email protected]>
To: <[email protected]>
Sent: Tuesday, March 29, 2005 7:11 PM
Subject: Re: [dods] DBTransactions and DBConnections


> Hi Sinisa,
>     Ah, looking the StandardDBTransaction class's source code, I see what
> you mean - DO aggregation (when it applies) is handled prior to handling
the
> auto-write option.  So in fact the intent of the if statement in the
> XxxQuery template was to provide consistent behaviour of the auto-write
> option, which makes sense - I didn't realise this was the case, as I
thought
> the .dontAggregateDOModifications() call would have triggered the .write()
> whenever the next operation was added to the DBTransaction, which probably
> would be sufficient (although not be the cleanest way to do it).
>     I had diagnosed the problem with inaccurate query results being due to
> the fact that querying within the transaction with the XxxQuery objects
(in
> Enhydra 5.1 at least) always performs the query on the database, although
I
> didn't realise it could pick up previously created XxxDO's from the
> transaction cache when it processes the results - thankfully, the case
where
> I had to diagnose this problem was the import of a set of data, so the
query
> within the transaction was actually returning no results at all (because
it
> was looking for the new, as-yet-uncommitted records added previously in
the
> transaction), which made the problem much easier to track down than it
would
> have been otherwise.  However, since the XxxQuery object (always?) queries
> the database in this case, and queued transactions could cause inaccurate
> results, might it make sense to drop the if condition entirely?  In all
> cases where I've been querying within a transaction, it has always been
the
> case that I expect that the operations of the transaction will influence
the
> results - otherwise (since I am using the dirtyDOs=3D"Compatible" mode) I
> would just use the default XxxQuery constructor and not pass it a
> DBTransaction so that the query gets executed outside of the transaction
> entirely.
>     If the XxxQuery objects always query the database when querying within
a
> DBTransaction (regardless of the dirtyDOs parameter or any other
> parameters), then I would advocate dropping the if statement, and have the
> .write() call triggered in the runQuery() method whether auto-write is on
or
> off - otherwise a query through the DBTransaction would return results
that
> could depend on when the DBTransaction last happened to call the .write()
> method.  For example, if the auto-write setting were eventually extended
to,
> say, have the DBTransaction call .write() every 'x' operations to keep its
> transaction queue from exceeding some maximum size, the query results
could
> even be unpredictable if the programmer forgets to explicitly call the
> .write() method on the DBTransaction before they make their query.  I
think
> it would be best to make the .write() call automatic in any cases where a
> XxxQuery object will be sending a query to the database to determine its
> results within a specific DBTransaction.
>
> Mike.
>
> ----- Original Message -----
> From: "Sinisa Milosevic" <[email protected]>
> To: <[email protected]>
> Sent: Tuesday, March 29, 2005 10:41 AM
> Subject: Re: [dods] DBTransactions and DBConnections
>
>
> Hi Mike,
>
> I think that we should keep current implementation and transaction.write()
> is necessary in xxxQuery class if autowrite is set to true. I'll try to
> explain you in few following sentences.
>
> If you use autowrite=3Dtrue, it doesn't mean that each modification will =
be
> stored in database immediately. In DODS we use DO aggregation, so changes
> will be written into database before next (different) DO will changed. All
> changes of the same DO will be aggregated in one modification. Because of
> that we must call transaction.write before query to be sure that each
> modification is stored in database.
>
> Regarding 'wrong' query results in transaction. Transaction cache is not
> Query cache, it contains only modified or created DOs in one transaction.
So
> query must be performed on database and results will be retrieved from
> transaction (or global) cache and/or database. That's a reason why you got
> 'wrong' results without using transaction.write() method after
modification
> in transaction.
>
> Regards,
> Sinisa
>
> ----- Original Message -----
> From: "Michael Strapp" <[email protected]>
> To: <[email protected]>
> Cc: "Sinisa Milosevic" <[email protected]>
> Sent: Monday, March 28, 2005 5:35 PM
> Subject: Re: [dods] DBTransactions and DBConnections
>
>
> > Hi again,
> >     Actually, having previously noticed a quirk when performing a query
> > within a transaction, I realise there's a bug in the code you copied
from
> > the XxxQuery object.  The section:
> >
> >
>
if(((com.lutris.appserver.server.sql.CachedDBTransaction)transaction).getAut
> oWrite())
> > try {
> >     transaction.write();
> > } catch (SQLException sqle) {
> >
> > Should actually have a not applied to the if condition.  I found that
when
> > you have auto-write off, and you use a XxxQuery object within the
> > transaction, the results can be incorrect due to the DBTransaction
wrapper
> > queueing up database operations that haven't been forwarded to the
> > database - I've been explicitly calling .write() prior to doing the
query
> in
> > such cases.  Turning auto-write on causes .write() to be called after
> every
> > operation - so the above .write() call is actually unnecessary when
> > auto-write is on (which was presumably the purpose of having the above
> > .write() call within an if statement).
> >
> > Sinisa, could you apply this fix to the runQuery() method in the code
> > templates for the XxxQuery objects?
> >
> > Thanks,
> > Mike.
> >
> > ----- Original Message -----
> > From: "Gilles Serasset" <[email protected]>
> > To: <[email protected]>
> > Sent: Thursday, March 24, 2005 3:31 AM
> > Subject: Re: [dods] DBTransactions and DBConnections
> >
> >
> > Hi Michael,
> >
> > Thanks for the suggestion, I'll try it if I cannot find another way.
> >
> > For the moment, I tried to implement the class as a subclass of a
> > Query, where I redefined the executeQuery method to fit my needs...
> >
> >      public ResultSet executeQuery(DBConnection conn) throws
> > SQLException {
> >          conn.execute(currentSQL);
> >          return null;
> >      }
> >
> > And I create an instance of the query using the trans as a parameter
> > (like other queries):
> >
> >      public SpecialDatabaseRequest(DBTransaction trans) {
> >          this.transaction =3D trans;
> >      }
> >
> > Invoking the query is done by creating an instance of the
> > SpecialDatabaseRequest and invoking the executeSql method:
> >
> >      public void executeSQL(String sql) throws DatabaseManagerException,
> > DataObjectException {
> >          //Flush the current transaction (?)
> >          currentSQL =3D sql;
> >          if ((transaction!=3Dnull) &&
> >              (transaction instanceof
> > com.lutris.appserver.server.sql.CachedDBTransaction)) {
> >
> > if(((com.lutris.appserver.server.sql.CachedDBTransaction)transaction).ge
> > tAutoWrite()) try {
> >                  transaction.write();
> >              } catch (SQLException sqle) {
> >                  sqle.printStackTrace();
> >                  throw new DataObjectException("Couldn't write
> > transaction: "+sqle);
> >              }
> >
> > //
> > ((com.lutris.appserver.server.sql.CachedDBTransaction)transaction).dontA
> > ggregateDOModifications();
> >          }
> >          // Create the DB Query Object
> >          try {
> >              if (transaction =3D=3D null) {
> >                  dbQuery =3D DODS.getDatabaseManager().createQuery();
> >              } else {
> >                  dbQuery =3D transaction.createQuery();
> >              }
> >          } catch ( DatabaseManagerException e ) {
> >              String err =3D "ERROR SpecialDatabaseRequest: Could not
> > create a DBQuery.  ";
> >              throw new DatabaseManagerException( err, e );
> >          }
> >          dbQuery.query( this );   // invokes executeQuery
> >          currentSQL =3D "";
> >      }
> >
> > The executeSQL code is inspired from code extracted from the Queries.
> > It hits DB without checking for anything in the caches. This may be
> > harmfull if caches are not in sync with DB and if the SQL does queries
> > on objects that have been modified before... (maybe a dods specialist
> > can confirm on this). However, my need are creation, truncation, drop
> > of tables so this is not a problem here...
> >
> > I still haven't tested it yet... but I'll tell you if this works...
> >
> > Regards,
> >
> > Gilles,
> >
> > --
> > Gilles S=E9rasset
> > GETA-CLIPS-IMAG (UJF, INPG & CNRS)
> > BP 53 - F-38041 Grenoble Cedex 9
> > Phone: +33 4 76 51 43 80
> > Fax:   +33 4 76 44 66 75
> >
> >
> >
> >
> >
> >
>
> --------------------------------------------------------------------------
>
>
>


----------------------------------------------------------------------------
----


>
> --
> You receive this message as a subscriber of the [email protected] mailing
list.
> To unsubscribe: mailto:[email protected]
> For general help: mailto:[email protected]?subject=3Dhelp
> ObjectWeb mailing lists service home page: http://www.objectweb.org/wws
>



------------=_1112202052-14665-163
Content-Type: text/plain; name="message.footer"
Content-Disposition: inline; filename="message.footer"
Content-Transfer-Encoding: 8bit


--
You receive this message as a subscriber of the [email protected] mailing list.
To unsubscribe: mailto:[email protected]
For general help: mailto:[email protected]?subject=help
ObjectWeb mailing lists service home page: http://www.objectweb.org/wws

------------=_1112202052-14665-163--