Re: question about added status
Qiwen Chen <[email protected]> Thu, 21 Feb 2019 09:04:13 -0800 (PST)
| Newsgroups | gmane.comp.web.zope.zodb |
|---|---|
| Message-ID | <[email protected]> |
------=_Part_635_618852687.1550768653746 Content-Type: multipart/alternative; boundary="----=_Part_636_184304138.1550768653747" ------=_Part_636_184304138.1550768653747 Content-Type: text/plain; charset="UTF-8" Thank you very much for the explanation. This is very helpful On Thursday, February 21, 2019 at 7:44:22 AM UTC-5, Jason Madden wrote: > > > > > On Feb 20, 2019, at 16:24, Qiwen Chen <[email protected] <javascript:>> > wrote: > > > > when an unsaved persistent object is set as sub-objects of another > persistent object with saved status, the unsaved persistent object becomes > Added. > > 'Added' is not a value that Persistent._p_status or Persistent._p_state > can have. Those can have the values 'unsaved', 'ghost', 'sticky', > 'changed', 'saved' or -1, 0, 1, 2, respectively ( > https://github.com/zopefoundation/persistent/blob/master/persistent/interfaces.py). > So I'm not quite sure what you mean. > > Is 'Added' an application level concept? Are you talking about being > present in the private attribute `_added` of a ZODB `Connection`? Or do you > mean "reachable in the object graph from the connection's root"? I'll > assume this last meaning because that's the most general (and is basically > what it means to a Connection). > > > However, when i check _p_jar, it is still None. > > That's correct. Simply because an object is reachable from a connection's > root doesn't mean its `_p_jar` has been set. Most commonly, that happens > automatically at transaction commit time, when a connection conceptually > traverses all objects reachable from its root and takes note of any added > or updated ones: > > py> import persistent, ZODB, ZODB.DemoStorage, transaction > py> db = ZODB.DB(ZODB.DemoStorage.DemoStorage()) > py> class O(persistent.Persistent): > ... pass > py> parent = conn.root.parent = O() > # Just because we're reachable doesn't mean we're saved or have a jar > py> parent._p_status > 'unsaved' > py> parent._p_jar > py> transaction.commit() > # Once we commit, the modified objects are added to the database and > connection > py> parent._p_status > 'saved' > py> parent._p_jar > <Connection at 10bc6ff20> > > # The same goes for a new child object added to a saved object > py> child = parent.child = O() > py> child._p_status > 'unsaved' > py> child._p_jar # None > py> transaction.commit() > py> child._p_status > 'saved' > py> child._p_jar > <Connection at 10bc6ff20> > > > > I will have to manually add the object to the connection in order to > populate _p_jar with such connection. > > I think that generally, manually adding objects to a connection should not > be necessary. There are cases in multi-database setups where an object is > initially reachable from multiple connections in different databases, and > which one to add it to is ambiguous; that case can require manually adding > an object to the correct database connection or you get an > InvalidObjectReference exception. But that's rare in my experience. > > > Is this a bug? > > No, I believe it's by design. Just because an object is *temporarily* > reachable doesn't mean that it will be reachable at connection commit time. > Prematurely adding it to a connection allocates an OID and generates > unnecessary garbage in that case: > > # Start fresh for clarity > py> db = ZODB.DB(ZODB.DemoStorage.DemoStorage()) > py> conn = db.open() > # Create an object, make it reachable from a database connection > py> parent = conn.root.parent = O() > # Explicitly add it now > py> conn.add(parent) > # This has the effect of allocating an OID for it, meaning it *will* be > stored > py> parent._p_oid > b'\x13\x03\xd7\xee\xe8yu\xd7' > > # Now go on and do some other work that results in "rolling back" this > object > py> conn.root.parent = None > py> transaction.commit() > py> conn.root.parent # No longer present > > # Yet this unreachable object was still stored in the database, > # where it will have to be garbage collected away > py> conn.get(b'\x13\x03\xd7\xee\xe8yu\xd7') > <__main__.O object at 0x10ba80a78 oid 0x1303d7eee87975d7 in <Connection at > 10bc15aa8>> > # it's visible to new connections, even though it's not reachable > py> conn2 = db.open() > py> conn2.root.parent # None > py> conn2.get(b'\x13\x03\xd7\xee\xe8yu\xd7') > <__main__.O object at 0x10bda8ef8 oid 0x1303d7eee87975d7 in <Connection at > 10be3c058>> > > # But if we pack the database... > py> db.pack() > # It is no longer visible to *new* connections > py> conn3 = db.open() > py> conn3.get(b'\x13\x03\xd7\xee\xe8yu\xd7') > Traceback (most recent call last) > ... > POSKeyError: 0x1303d7eee87975d7 > > # Very confusingly, it is still visible from connections that had it > cached > py> conn3.close() > py> conn2.close() > py> conn.close() > py> conn = db.open() > py> conn2 = db.open() > py> conn3 = db.open() > py> conn.get(b'\x13\x03\xd7\xee\xe8yu\xd7') > <__main__.O object at 0x10ba80a78 oid 0x1303d7eee87975d7 in <Connection at > 10bc15aa8>> > py> conn2.get(b'\x13\x03\xd7\xee\xe8yu\xd7') > <__main__.O object at 0x10bda8ef8 oid 0x1303d7eee87975d7 in <Connection at > 10be3c058>> > py> conn3.get(b'\x13\x03\xd7\xee\xe8yu\xd7') > Traceback (most recent call last) > ... > POSKeyError: 0x1303d7eee87975d7 > > > Is there a reliable way to check if an object is added or not? > > I suppose it's application specific. In an application using > zope.container and zope.location, you could check that there's a traversal > path back to the root; zope.keyreference's IConnection adapter can also be > helpful. > > I don't think ZODB offers a shortcut to find out "is this object reachable > from the connection's root" before commit time. > > ~Jason -- You received this message because you are subscribed to the Google Groups "zodb" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout. ------=_Part_636_184304138.1550768653747 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">Thank you very much for the explanation. This is very help= ful<br><br>On Thursday, February 21, 2019 at 7:44:22 AM UTC-5, Jason Madden= wrote:<blockquote class=3D"gmail_quote" style=3D"margin: 0;margin-left: 0.= 8ex;border-left: 1px #ccc solid;padding-left: 1ex;"> <br> <br>> On Feb 20, 2019, at 16:24, Qiwen Chen <<a href=3D"javascript:" = target=3D"_blank" gdf-obfuscated-mailto=3D"jnAIh_i3AAAJ" rel=3D"nofollow" o= nmousedown=3D"this.href=3D'javascript:';return true;" onclick=3D"th= is.href=3D'javascript:';return true;">[email protected]</a>> wrot= e: <br>>=20 <br>> when an unsaved persistent object is set as sub-objects of another= persistent object with saved status, the unsaved persistent object becomes= Added. <br> <br>'Added' is not a value that Persistent._p_status or Persistent.= _p_state can have. Those can have the values 'unsaved', 'ghost&= #39;, 'sticky', 'changed', 'saved' or -1, 0, 1, 2, = respectively (<a href=3D"https://github.com/zopefoundation/persistent/blob/= master/persistent/interfaces.py" target=3D"_blank" rel=3D"nofollow" onmouse= down=3D"this.href=3D'https://www.google.com/url?q\x3dhttps%3A%2F%2Fgith= ub.com%2Fzopefoundation%2Fpersistent%2Fblob%2Fmaster%2Fpersistent%2Finterfa= ces.py\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFEkDir_eEchV38VBc5nzlqHVEUug= ';return true;" onclick=3D"this.href=3D'https://www.google.com/url?= q\x3dhttps%3A%2F%2Fgithub.com%2Fzopefoundation%2Fpersistent%2Fblob%2Fmaster= %2Fpersistent%2Finterfaces.py\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFEkDi= r_eEchV38VBc5nzlqHVEUug';return true;">https://github.com/<wbr>zopefoun= dation/persistent/<wbr>blob/master/persistent/<wbr>interfaces.py</a>). So I= 'm not quite sure what you mean.=20 <br> <br>Is 'Added' an application level concept? Are you talking about = being present in the private attribute `_added` of a ZODB `Connection`? Or = do you mean "reachable in the object graph from the connection's r= oot"? I'll assume this last meaning because that's the most ge= neral (and is basically what it means to a Connection). <br> <br>> However, when i check _p_jar, it is still None. <br> <br>That's correct. Simply because an object is reachable from a connec= tion's root doesn't mean its `_p_jar` has been set. Most commonly, = that happens automatically at transaction commit time, when a connection co= nceptually traverses all =C2=A0objects reachable from its root and takes no= te of any added or updated ones: <br> <br>py> import persistent, ZODB, ZODB.DemoStorage, transaction <br>py> db =3D ZODB.DB(ZODB.DemoStorage.<wbr>DemoStorage()) <br>py> class O(persistent.Persistent):=20 <br>... =C2=A0 =C2=A0 pass <br>py> parent =3D conn.root.parent =3D O() <br># Just because we're reachable doesn't mean we're saved or = have a jar <br>py> parent._p_status <br>'unsaved' <br>py> parent._p_jar <br>py> transaction.commit() <br># Once we commit, the modified objects are added to the database and co= nnection <br>py> parent._p_status <br>'saved' <br>py> parent._p_jar <br><Connection at 10bc6ff20> <br> <br># The same goes for a new child object added to a saved object <br>py> child =3D parent.child =3D O() <br>py> child._p_status <br>'unsaved' <br>py> child._p_jar # None <br>py> transaction.commit() <br>py> child._p_status <br>'saved' <br>py> child._p_jar <br><Connection at 10bc6ff20> <br> <br> <br>> I will have to manually add the object to the connection in order = to populate _p_jar with such connection. <br> <br>I think that generally, manually adding objects to a connection should = not be necessary. There are cases in multi-database setups where an object = is initially reachable from multiple connections in different databases, an= d which one to add it to is ambiguous; that case can require manually addin= g an object to the correct database connection or you get an InvalidObjectR= eference exception. But that's rare in my experience. <br> <br>> Is this a bug? <br> <br>No, I believe it's by design. Just because an object is *temporaril= y* reachable doesn't mean that it will be reachable at connection commi= t time. Prematurely adding it to a connection allocates an OID and generate= s unnecessary garbage in that case: <br> <br># Start fresh for clarity <br>py> db =3D ZODB.DB(ZODB.DemoStorage.<wbr>DemoStorage()) <br>py> conn =3D db.open() <br># Create an object, make it reachable from a database connection <br>py> parent =3D conn.root.parent =3D O() <br># Explicitly add it now <br>py> conn.add(parent) <br># This has the effect of allocating an OID for it, meaning it *will* be= stored <br>py> parent._p_oid <br>b'\x13\x03\xd7\xee\xe8yu\xd7' <br> <br># Now go on and do some other work that results in "rolling back&q= uot; this object <br>py> conn.root.parent =3D None <br>py> transaction.commit() <br>py> conn.root.parent # No longer present <br> <br># Yet this unreachable object was still stored in the database, <br># where it will have to be garbage collected away <br>py> conn.get(b'\x13\x03\xd7\xee\<wbr>xe8yu\xd7') <br><__main__.O object at 0x10ba80a78 oid 0x1303d7eee87975d7 in <Conn= ection at 10bc15aa8>> <br># it's visible to new connections, even though it's not reachab= le <br>py> conn2 =3D db.open() <br>py> conn2.root.parent # None <br>py> conn2.get(b'\x13\x03\xd7\xee\<wbr>xe8yu\xd7') <br><__main__.O object at 0x10bda8ef8 oid 0x1303d7eee87975d7 in <Conn= ection at 10be3c058>> <br> <br># But if we pack the database... <br>py> db.pack() <br># It is no longer visible to *new* connections <br>py> conn3 =3D db.open() <br>py> conn3.get(b'\x13\x03\xd7\xee\<wbr>xe8yu\xd7') <br>Traceback (most recent call last) <br>... <br>POSKeyError: 0x1303d7eee87975d7 <br> <br># Very confusingly, it is still visible from connections that had it ca= ched <br>py> conn3.close() <br>py> conn2.close() <br>py> conn.close() <br>py> conn =3D db.open() <br>py> conn2 =3D db.open() <br>py> conn3 =3D db.open() <br>py> conn.get(b'\x13\x03\xd7\xee\<wbr>xe8yu\xd7') <br><__main__.O object at 0x10ba80a78 oid 0x1303d7eee87975d7 in <Conn= ection at 10bc15aa8>> <br>py> conn2.get(b'\x13\x03\xd7\xee\<wbr>xe8yu\xd7') <br><__main__.O object at 0x10bda8ef8 oid 0x1303d7eee87975d7 in <Conn= ection at 10be3c058>> <br>py> conn3.get(b'\x13\x03\xd7\xee\<wbr>xe8yu\xd7') <br>Traceback (most recent call last) <br>... <br>POSKeyError: 0x1303d7eee87975d7 <br> <br>> Is there a reliable way to check if an object is added or not? <br> <br>I suppose it's application specific. In an application using zope.c= ontainer and zope.location, you could check that there's a traversal pa= th back to the root; zope.keyreference's IConnection adapter can also b= e helpful. <br> <br>I don't think ZODB offers a shortcut to find out "is this obje= ct reachable from the connection's root" before commit time. <br> <br>~Jason</blockquote></div> <p></p> -- <br /> You received this message because you are subscribed to the Google Groups &= quot;zodb" group.<br /> To unsubscribe from this group and stop receiving emails from it, send an e= mail to <a href=3D"mailto:[email protected]">zodb+unsubscri= [email protected]</a>.<br /> For more options, visit <a href=3D"https://groups.google.com/d/optout">http= s://groups.google.com/d/optout</a>.<br /> ------=_Part_636_184304138.1550768653747-- ------=_Part_635_618852687.1550768653746--