Re: Databases and threading
Christopher Lozinski <[email protected]> Tue, 19 Nov 2019 14:05:46 +0100
| Newsgroups | gmane.comp.web.zope.zodb |
|---|---|
| Message-ID | <[email protected]> |
--Apple-Mail=_7C5C9C4E-AA70-48A7-8C80-A35CB31205D4 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" It is an interesting question.=20 I am running uwsgi. That supports multiple threads. =20 In the configuration file, I specify how many threads to support.=20 How does uwsgi do it? I do not know. So I read the source.=20 When my applications starts it reads a config file, and creates a db object= .=20 On each request uwsgi opens a new connection. I am using the cromlech fram= ework.=20 https://github.com/Cromlech/cromlech.zodb/blob/b91d3b7b384cd107e7ea8423f485= 0fd5fd6749ad/src/cromlech/zodb/middleware.py#L38 <https://github.com/Cromle= ch/cromlech.zodb/blob/b91d3b7b384cd107e7ea8423f4850fd5fd6749ad/src/cromlech= /zodb/middleware.py#L38> So I think that if you open a new connection in each thread, you should be = fine.=20 Furthermore, I think your code has the problem, that you have to first crea= te a connection,=20 before trying to invoke transaction machinery. =20 Hope that helps.=20 Did I get that right? Chris > On Nov 19, 2019, at 11:44 AM, =C3=89loi Rivard <[email protected]> wrote: >=20 > Hello, There are a few things I wonder about ZODB and concurrency, and I = am not sure of the answers the documentation gives. > The documentation states tha database connections, transactions managers = and transactions are not thread safe. It also states that we cannot share d= atabases between processes. But can we share databases an storages between = threads? Is this code safe? >=20 > import ZODB, threading > db =3D ZODB.DB(None) > def work(): > with db.transaction() as conn: > do_stuff() >=20 > threading.Thread(target=3Dwork).start() > threading.Thread(target=3Dwork).start() >=20 > Thank you for your answers >=20 > =C3=89loi >=20 > --=20 > You received this message because you are subscribed to the Google Groups= "zodb" group. > To unsubscribe from this group and stop receiving emails from it, send an= email to [email protected] <mailto:zodb+unsubscribe@google= groups.com>. > To view this discussion on the web visit https://groups.google.com/d/msgi= d/zodb/590aa292-4e00-4f26-94a0-8f02f36388fa%40googlegroups.com <https://gro= ups.google.com/d/msgid/zodb/590aa292-4e00-4f26-94a0-8f02f36388fa%40googlegr= oups.com?utm_medium=3Demail&utm_source=3Dfooter>. --=20 You received this message because you are subscribed to the Google Groups "= zodb" group. To unsubscribe from this group and stop receiving emails from it, send an e= mail to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/= zodb/CC2A0684-4CFE-41D3-8822-BBF4041378C8%40specialtyjobmarkets.com. --Apple-Mail=_7C5C9C4E-AA70-48A7-8C80-A35CB31205D4 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset="UTF-8" <html><body style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; -webk= it-line-break: after-white-space;" class=3D""><font size=3D"2" class=3D"">I= t is an interesting question. </font><div class=3D""><font size=3D"2" = class=3D""><br class=3D""></font></div><div class=3D""><font size=3D"2" cla= ss=3D"">I am running uwsgi. That supports multiple threads. </f= ont></div><div class=3D""><font size=3D"2" class=3D""><br class=3D""></font= ></div><div class=3D""><font size=3D"2" class=3D"">In the configuration fil= e, I specify how many threads to support. </font></div><div class=3D""= ><font size=3D"2" class=3D""><br class=3D""></font></div><div class=3D""><f= ont size=3D"2" class=3D"">How does uwsgi do it? I do not know. &= nbsp;So I read the source. </font></div><div class=3D""><font size=3D"= 2" class=3D""><br class=3D""></font></div><div class=3D""><font size=3D"2" = class=3D"">When my applications starts it reads a config file, and creates = a db object. </font></div><div class=3D""><font size=3D"2" class= =3D"">On each request uwsgi opens a new connection. I am using&n= bsp;the cromlech framework. </font></div><div class=3D""><font size=3D= "2" class=3D""><br class=3D""></font></div><div class=3D""><a href=3D"https= ://github.com/Cromlech/cromlech.zodb/blob/b91d3b7b384cd107e7ea8423f4850fd5f= d6749ad/src/cromlech/zodb/middleware.py#L38" class=3D""><font size=3D"2" cl= ass=3D"">https://github.com/Cromlech/cromlech.zodb/blob/b91d3b7b384cd107e7e= a8423f4850fd5fd6749ad/src/cromlech/zodb/middleware.py#L38</font></a></div><= div class=3D""><font size=3D"2" class=3D""><br class=3D""></font></div><div= class=3D""><font size=3D"2" class=3D"">So I think that if you open a new c= onnection in each thread, you should be fine. </font></div><div class= =3D""><font size=3D"2" class=3D""><br class=3D""></font></div><div class=3D= ""><font size=3D"2" class=3D"">Furthermore, I think your code has the probl= em, that you have to first create a connection, </font></div><div= class=3D""><font size=3D"2" class=3D"">before trying to invoke transaction= machinery. </font></div><div class=3D""><font size=3D"2" class=3D"">= <br class=3D""></font></div><div class=3D""><font size=3D"2" class=3D"">Hop= e that helps. </font></div><div class=3D""><font size=3D"2" class=3D""= >Did I get that right?</font></div><div class=3D""><font size=3D"2" class= =3D""><br class=3D""></font></div><div class=3D""><font size=3D"2" class=3D= "">Chris</font></div><div class=3D""><font size=3D"2" class=3D""><br class= =3D""></font><div><font size=3D"2" class=3D""><br class=3D""></font><blockq= uote type=3D"cite" class=3D""><div class=3D""><font size=3D"2" class=3D"">O= n Nov 19, 2019, at 11:44 AM, =C3=89loi Rivard <<a href=3D"mailto:azmeuk@= gmail.com" class=3D"">[email protected]</a>> wrote:</font></div><font siz= e=3D"2" class=3D""><br class=3D"Apple-interchange-newline"></font><div clas= s=3D""><div dir=3D"ltr" class=3D""><div class=3D""><font size=3D"2" class= =3D"">Hello, There are a few things I wonder about ZODB and concurrency, an= d I am not sure of the answers the documentation gives.</font></div><div cl= ass=3D""><font size=3D"2" class=3D"">The documentation states tha database = connections, transactions managers and transactions are not thread safe. It= also states that we cannot share databases between processes. But can we s= hare databases an storages between threads? Is this code safe?<br class=3D"= "></font></div><div class=3D""><font size=3D"2" class=3D""><br class=3D""><= /font></div><div class=3D""><div class=3D"prettyprint" data-darkreader-inli= ne-bgcolor=3D"" data-darkreader-inline-border-top=3D"" data-darkreader-inli= ne-border-right=3D"" data-darkreader-inline-border-bottom=3D"" data-darkrea= der-inline-border-left=3D"" style=3D"background-color: rgb(250, 250, 250); = border: 1px solid rgb(187, 187, 187); overflow-wrap: break-word; --darkread= er-inline-bgcolor: #1a1b1c; --darkreader-inline-border-top: #414141; --dark= reader-inline-border-right: #414141; --darkreader-inline-border-bottom: #41= 4141; --darkreader-inline-border-left: #414141;"><code class=3D"prettyprint= "><div class=3D"subprettyprint"><font size=3D"3" class=3D""><span class=3D"= styled-by-prettify" style=3D"color: rgb(0, 0, 136);">import</span><span cla= ss=3D"styled-by-prettify"> ZODB</span><span class=3D"styled-by-prettif= y" style=3D"color: rgb(102, 102, 0);">,</span><span class=3D"styled-by-pret= tify"> threading<br class=3D"">db </span><span class=3D"styled-by= -prettify" style=3D"color: rgb(102, 102, 0);">=3D</span><span class=3D"styl= ed-by-prettify"> ZODB</span><span class=3D"styled-by-prettify" style= =3D"color: rgb(102, 102, 0);">.</span><span class=3D"styled-by-prettify">DB= </span><span class=3D"styled-by-prettify" style=3D"color: rgb(102, 102, 0);= ">(</span><span class=3D"styled-by-prettify" style=3D"color: rgb(0, 0, 136)= ;">None</span><span class=3D"styled-by-prettify" style=3D"color: rgb(102, 1= 02, 0);">)</span><span class=3D"styled-by-prettify"><br class=3D""></span><= span class=3D"styled-by-prettify" style=3D"color: rgb(0, 0, 136);">def</spa= n><span class=3D"styled-by-prettify"> work</span><span class=3D"styled= -by-prettify" style=3D"color: rgb(102, 102, 0);">():</span><span class=3D"s= tyled-by-prettify"><br class=3D""> </span><span class=3D"= styled-by-prettify" style=3D"color: rgb(0, 0, 136);">with</span><span class= =3D"styled-by-prettify"> db</span><span class=3D"styled-by-prettify" s= tyle=3D"color: rgb(102, 102, 0);">.</span><span class=3D"styled-by-prettify= ">transaction</span><span class=3D"styled-by-prettify" style=3D"color: rgb(= 102, 102, 0);">()</span><span class=3D"styled-by-prettify"> </span><sp= an class=3D"styled-by-prettify" style=3D"color: rgb(0, 0, 136);">as</span><= span class=3D"styled-by-prettify"> conn</span><span class=3D"styled-by= -prettify" style=3D"color: rgb(102, 102, 0);">:</span><span class=3D"styled= -by-prettify"><br class=3D""> do_stuff</span><sp= an class=3D"styled-by-prettify" style=3D"color: rgb(102, 102, 0);">()</span= ><span class=3D"styled-by-prettify"><br class=3D""><br class=3D"">threading= </span><span class=3D"styled-by-prettify" style=3D"color: rgb(102, 102, 0);= ">.</span><span class=3D"styled-by-prettify" style=3D"color: rgb(102, 0, 10= 2);">Thread</span><span class=3D"styled-by-prettify" style=3D"color: rgb(10= 2, 102, 0);">(</span><span class=3D"styled-by-prettify">target</span><span = class=3D"styled-by-prettify" style=3D"color: rgb(102, 102, 0);">=3D</span><= span class=3D"styled-by-prettify">work</span><span class=3D"styled-by-prett= ify" style=3D"color: rgb(102, 102, 0);">).</span><span class=3D"styled-by-p= rettify">start</span><span class=3D"styled-by-prettify" style=3D"color: rgb= (102, 102, 0);">()</span><span class=3D"styled-by-prettify"><br class=3D"">= threading</span><span class=3D"styled-by-prettify" style=3D"color: rgb(102,= 102, 0);">.</span><span class=3D"styled-by-prettify" style=3D"color: rgb(1= 02, 0, 102);">Thread</span><span class=3D"styled-by-prettify" style=3D"colo= r: rgb(102, 102, 0);">(</span><span class=3D"styled-by-prettify">target</sp= an><span class=3D"styled-by-prettify" style=3D"color: rgb(102, 102, 0);">= =3D</span><span class=3D"styled-by-prettify">work</span><span class=3D"styl= ed-by-prettify" style=3D"color: rgb(102, 102, 0);">).</span><span class=3D"= styled-by-prettify">start</span><span class=3D"styled-by-prettify" style=3D= "color: rgb(102, 102, 0);">()</span><span class=3D"styled-by-prettify"><br = class=3D""></span></font></div></code></div></div><div class=3D""><font siz= e=3D"2" class=3D""><br class=3D""></font></div><div class=3D""><font size= =3D"2" class=3D"">Thank you for your answers</font></div><div class=3D""><f= ont size=3D"2" class=3D""><br class=3D""></font></div><div class=3D""><font= size=3D"2" class=3D"">=C3=89loi<br class=3D""></font></div></div><div clas= s=3D""><font size=3D"2" class=3D""><br class=3D"webkit-block-placeholder"><= /font></div><font size=3D"2" class=3D"">-- <br class=3D"">You received= this message because you are subscribed to the Google Groups "zodb" group.= <br class=3D"">To unsubscribe from this group and stop receiving emails fro= m it, send an email to <a href=3D"mailto:zodb+unsubscribe@googlegroups= .com" class=3D"">[email protected]</a>.<br class=3D"">To vi= ew this discussion on the web visit <a href=3D"https://groups.google.c= om/d/msgid/zodb/590aa292-4e00-4f26-94a0-8f02f36388fa%40googlegroups.com?utm= _medium=3Demail&utm_source=3Dfooter" class=3D"">https://groups.google.c= om/d/msgid/zodb/590aa292-4e00-4f26-94a0-8f02f36388fa%40googlegroups.com</a>= .<br class=3D""></font></div></blockquote><div class=3D""><div class=3D""><= font size=3D"2" class=3D""><br class=3D""></font></div></div></div></div></= body></html> <p></p> -- <br /> You received this message because you are subscribed to the Google Groups &= quot;zodb" group.<br /> To unsubscribe from this group and stop receiving emails from it, send an e= mail to <a href=3D"mailto:[email protected]">zodb+unsubscri= [email protected]</a>.<br /> To view this discussion on the web visit <a href=3D"https://groups.google.c= om/d/msgid/zodb/CC2A0684-4CFE-41D3-8822-BBF4041378C8%40specialtyjobmarkets.= com?utm_medium=3Demail&utm_source=3Dfooter">https://groups.google.com/d/msg= id/zodb/CC2A0684-4CFE-41D3-8822-BBF4041378C8%40specialtyjobmarkets.com</a>.= <br /> --Apple-Mail=_7C5C9C4E-AA70-48A7-8C80-A35CB31205D4--