Re: Issue with transactions...

> On Jun 17, 2020, at 05:16, [email protected] wrote:
>=20
> I have noticed that during a process that the memory usage is up to 13 to=
 14 GB.
>=20
> I tried using gc.collect() which did nothing. So I tried a "transaction.a=
bort()" and most of the memory was returned after about 2 to 3 minutes.
>=20
> This process does not change anything within the persistent objects so wh=
y is it adding those persistent objects to a transaction? I have added a "t=
ransaction.abort()" to the process and the memory stays around 370 MB.
>=20

It's not adding them to a transaction.=20

ZODB Connections cache persistent objects. This is for performance, and in =
the case of modifications, to ensure that all the objects accessed come fro=
m one consistent view of the database as-of the time the transaction began.=
 Until the transaction ends, and the view of the database is updated, persi=
stent objects that the connection accesses remain in the connection's cache=
.

At transaction boundaries, the Connection automatically performs cache garb=
age collection, removing unmodified persistent objects from the cache as ne=
cessary to keep the cache within configured limits.=20

By calling `transaction.abort()`, you are creating a transaction boundary: =
not only are you freeing objects from the cache, you are also updating the =
view of the database to incorporate whatever recent changes may have been c=
ommitted, which may be inconsistent with the previous view of the database =
you were working with.

It's good practice to explicitly delineate transaction boundaries in your a=
pplication: units of work that need a consistent view of the database. In a=
 server or web application, transaction boundaries usually correspond to cl=
ient requests. At the beginning of processing the request, call `transactio=
n.begin()` to update the view of the database, and at the end, call `transa=
ction.commit()` (or `abort()`) to free (cached) resources.

If you do nat have natural transaction boundaries in your application, you =
can manually ask the Connection to remove un-modified objects from its cach=
e at appropriate intervals using `Connection.cacheGC()`[1] (or `cacheMinimi=
ze()`, but that negates the point of having a cache entirely). Unlike arbit=
rarily committing or aborting a transaction during processing, this won't u=
pdate the view of the database, meaning the objects stay consistent with pr=
eviously seen objects. Note, however, that on certain storage backends such=
 as RelStorage, long-running transactions can become expensive if there are=
 other transactions writing to the database.

[1] https://github.com/zopefoundation/ZODB/blob/22df1fd1141ab0c9db2a163799b=
87fe3585f15ef/src/ZODB/interfaces.py#L170

~Jason

--=20
You received this message because you are subscribed to the Google Groups "=
zodb" group.
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to [email protected].
To view this discussion on the web visit https://groups.google.com/d/msgid/=
zodb/D7968136-E3C1-434E-B2AE-E1649E15330B%40nextthought.com.