Session file locking not working properly in multithreaded environment

Ralph Heinkel <ralph.heinkel-S0/[email protected]> Fri, 10 May 2019 05:08:46 -0700 (PDT)
Newsgroups	gmane.comp.python.cherrypy
Message-ID	<[email protected]>
------=_Part_863_848813383.1557490126828
Content-Type: multipart/alternative; 
	boundary="----=_Part_864_1312001152.1557490126830"

------=_Part_864_1312001152.1557490126830
Content-Type: text/plain; charset="UTF-8"

I have a very simple web-page which makes 20 (parallel) connections to my 
cherrypy server via <img> tags targetting images which are calculated 
programmatically on-the-fly when accessed:

<html>
<img src="/CP/getPlot?image_id=i_1"/>
<img src="/CP/getPlot?image_id=i_2"/>
[.. 18 more entries ]
<img src="/CP/getPlot?image_id=i_20"/>
</html>

In some rare, random cases some of the 'getPlot' requests fail  because 
cherrypy receives an incomplete session object from a session file, i.e. 
data that cannot be unpickled. In such a case cherrypy creates a new 
session object. Since authentication is bound to the session, the request 
then fails due to missing privileges, and hence all followup requests to 
'getPlot' fail as well.

When digging in the sessions and lockfile code of cherrypy I added some log 
statements to see in which order things happen when a session is opened and 
closed. This way I found that the same lock file was locked simultaneously 
by different parallel threads. This of course leads to the case where a 
session was partially written by one thread while another thread started 
reading from it, causing it to receive incomplete session data, and hence 
unpickle failed.

The log reveilled what is happening:

UnixLockFile._lock_file called for /fs/local/run/sessions/session-
3146212444067902ec35e41a4af0db64bcbacf4b.lock on fileno 32 (thread=
139877983168256)
Loaded session with data: ({'csrf_token': 
'c8cc04fa-6cec-4dc2-a690-d3d1398f6322', 'login': 'rh'}, datetime.datetime(
2019, 5, 10, 19, 29, 26, 591346))
saving session data: dict_items([('csrf_token', 
'c8cc04fa-6cec-4dc2-a690-d3d1398f6322'), ('login', 'rh')])
UnixLockFile.release called for /fs/local/run/sessions/session-
3146212444067902ec35e41a4af0db64bcbacf4b.lock (thread=139877983168256)
2019-05-10 11:29:27,141:pid=22163:127.0.0.1 - rh300070 [10/May/2019:11:29:27
] "GET /CP/getPlot?image_id=i_1 HTTP/1.1" 200 14708 
"http://localhost:8080/CP/static/quant.html" 

UnixLockFile._lock_file called for /fs/local/run/sessions/session-
3146212444067902ec35e41a4af0db64bcbacf4b.lock on fileno 36 (thread=
139877479868160)
Loaded session with data: ({'csrf_token': 
'c8cc04fa-6cec-4dc2-a690-d3d1398f6322', 'login': 'rh'}, datetime.datetime(
2019, 5, 10, 19, 29, 27, 146440))
saving session data: dict_items([('csrf_token', 
'c8cc04fa-6cec-4dc2-a690-d3d1398f6322'), ('login', 'rh')])
UnixLockFile.release called for /fs/local/run/sessions/session-
3146212444067902ec35e41a4af0db64bcbacf4b.lock (thread=139877479868160)
2019-05-10 11:29:27,153:pid=22163:127.0.0.1 - rh300070 [10/May/2019:11:29:27
] "GET /CP/getPlot?image_id=i_7 HTTP/1.1" 200 14708 
"http://localhost:8080/CP/static/quant.html"

UnixLockFile._lock_file called for /fs/local/run/sessions/session-
3146212444067902ec35e41a4af0db64bcbacf4b.lock on fileno 32 (thread=
139877454690048)
UnixLockFile._lock_file called for /fs/local/run/sessions/session-
3146212444067902ec35e41a4af0db64bcbacf4b.lock on fileno 36 (thread=
139877479868160)
[...]

The first two requests looked entirely ok, the lock was acquired, the 
session was loaded, then after the request the session got saved, and the 
lock released. 

But then suddenly two threads obtain a lock to the same file! How is this 
possible? I have to mention that this does not happen every time I load the 
main page, sometimes I have to reload 30 or 40 times until this behavior 
shows up.

Looking at the code I have no idea how this can happen, the code (it uses 
fcntl.flock() with correct flags) looks entirely OK to me. 

Any help is very much appreciated.

PS: I work under Centos7, Python 3.6.4, cherrypy ~=15.0.0 (but also the 
latest 18.x.x version of cherrypy shows the same behavior)

-- 
You received this message because you are subscribed to the Google Groups "cherrypy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cherrypy-users+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/[email protected]
To post to this group, send email to cherrypy-users-/JYPxA39Uh5TLH3MbocFF+G/[email protected]
Visit this group at https://groups.google.com/group/cherrypy-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/cherrypy-users/5460092d-e7e7-49a3-974d-d3cdc1c05885%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

------=_Part_864_1312001152.1557490126830
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">I have a very simple web-page which makes 20 (parallel) co=
nnections to my cherrypy server via  &lt;img&gt; tags targetting images whi=
ch are calculated programmatically on-the-fly when accessed:<br><br><div st=
yle=3D"background-color: rgb(250, 250, 250); border-color: rgb(187, 187, 18=
7); border-style: solid; border-width: 1px; overflow-wrap: break-word;" cla=
ss=3D"prettyprint"><code class=3D"prettyprint"><div class=3D"subprettyprint=
"><span style=3D"color: #008;" class=3D"styled-by-prettify">&lt;html&gt;</s=
pan><span style=3D"color: #000;" class=3D"styled-by-prettify"><br></span><s=
pan style=3D"color: #008;" class=3D"styled-by-prettify">&lt;img</span><span=
 style=3D"color: #000;" class=3D"styled-by-prettify"> </span><span style=3D=
"color: #606;" class=3D"styled-by-prettify">src</span><span style=3D"color:=
 #660;" class=3D"styled-by-prettify">=3D</span><span style=3D"color: #080;"=
 class=3D"styled-by-prettify">&quot;/CP/getPlot?image_id=3Di_1&quot;</span>=
<span style=3D"color: #008;" class=3D"styled-by-prettify">/&gt;</span><span=
 style=3D"color: #000;" class=3D"styled-by-prettify"><br></span><span style=
=3D"color: #008;" class=3D"styled-by-prettify">&lt;img</span><span style=3D=
"color: #000;" class=3D"styled-by-prettify"> </span><span style=3D"color: #=
606;" class=3D"styled-by-prettify">src</span><span style=3D"color: #660;" c=
lass=3D"styled-by-prettify">=3D</span><span style=3D"color: #080;" class=3D=
"styled-by-prettify">&quot;/CP/getPlot?image_id=3Di_2&quot;</span><span sty=
le=3D"color: #008;" class=3D"styled-by-prettify">/&gt;</span><span style=3D=
"color: #000;" class=3D"styled-by-prettify"><br>[.. 18 more entries ]<br></=
span><span style=3D"color: #008;" class=3D"styled-by-prettify">&lt;img</spa=
n><span style=3D"color: #000;" class=3D"styled-by-prettify"> </span><span s=
tyle=3D"color: #606;" class=3D"styled-by-prettify">src</span><span style=3D=
"color: #660;" class=3D"styled-by-prettify">=3D</span><span style=3D"color:=
 #080;" class=3D"styled-by-prettify">&quot;/CP/getPlot?image_id=3Di_20&quot=
;</span><span style=3D"color: #008;" class=3D"styled-by-prettify">/&gt;</sp=
an><span style=3D"color: #000;" class=3D"styled-by-prettify"><br></span><sp=
an style=3D"color: #008;" class=3D"styled-by-prettify">&lt;/html&gt;</span>=
</div></code></div><br>In some rare, random cases some of the &#39;getPlot&=
#39; requests fail=C2=A0 because cherrypy receives an incomplete session ob=
ject from a session file, i.e. data that cannot be unpickled. In such a cas=
e cherrypy creates a new session object. Since authentication is bound to t=
he session, the request then fails due to missing privileges, and hence all=
 followup requests to &#39;getPlot&#39; fail as well.<br><br>When digging i=
n the sessions and lockfile code of cherrypy I added some log statements to=
 see in which order things happen when a session is opened and closed. This=
 way I found that the same lock file was locked simultaneously by different=
 parallel threads. This of course leads to the case where a session was par=
tially written by one thread while another thread started reading from it, =
causing it to receive incomplete session data, and hence unpickle failed.<b=
r><br>The log reveilled what is happening:<br><br><div style=3D"background-=
color: rgb(250, 250, 250); border-color: rgb(187, 187, 187); border-style: =
solid; border-width: 1px; overflow-wrap: break-word;" class=3D"prettyprint"=
><code class=3D"prettyprint"><div class=3D"subprettyprint"><span style=3D"c=
olor: #606;" class=3D"styled-by-prettify">UnixLockFile</span><span style=3D=
"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"color: #=
000;" class=3D"styled-by-prettify">_lock_file called </span><span style=3D"=
color: #008;" class=3D"styled-by-prettify">for</span><span style=3D"color: =
#000;" class=3D"styled-by-prettify"> </span><span style=3D"color: #660;" cl=
ass=3D"styled-by-prettify">/</span><span style=3D"color: #000;" class=3D"st=
yled-by-prettify">fs</span><span style=3D"color: #660;" class=3D"styled-by-=
prettify">/</span><span style=3D"color: #008;" class=3D"styled-by-prettify"=
>local</span><span style=3D"color: #660;" class=3D"styled-by-prettify">/</s=
pan><span style=3D"color: #000;" class=3D"styled-by-prettify">run</span><sp=
an style=3D"color: #660;" class=3D"styled-by-prettify">/</span><span style=
=3D"color: #000;" class=3D"styled-by-prettify">sessions</span><span style=
=3D"color: #660;" class=3D"styled-by-prettify">/</span><span style=3D"color=
: #000;" class=3D"styled-by-prettify">session</span><span style=3D"color: #=
660;" class=3D"styled-by-prettify">-</span><span style=3D"color: #066;" cla=
ss=3D"styled-by-prettify">3146212444067902ec35e41a4af0db64bcbacf4b</span><s=
pan style=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=
=3D"color: #008;" class=3D"styled-by-prettify">lock</span><span style=3D"co=
lor: #000;" class=3D"styled-by-prettify"> on fileno </span><span style=3D"c=
olor: #066;" class=3D"styled-by-prettify">32</span><span style=3D"color: #0=
00;" class=3D"styled-by-prettify"> </span><span style=3D"color: #660;" clas=
s=3D"styled-by-prettify">(</span><span style=3D"color: #000;" class=3D"styl=
ed-by-prettify">thread</span><span style=3D"color: #660;" class=3D"styled-b=
y-prettify">=3D</span><span style=3D"color: #066;" class=3D"styled-by-prett=
ify">139877983168256</span><span style=3D"color: #660;" class=3D"styled-by-=
prettify">)</span><span style=3D"color: #000;" class=3D"styled-by-prettify"=
><br></span><span style=3D"color: #606;" class=3D"styled-by-prettify">Loade=
d</span><span style=3D"color: #000;" class=3D"styled-by-prettify"> session =
</span><span style=3D"color: #008;" class=3D"styled-by-prettify">with</span=
><span style=3D"color: #000;" class=3D"styled-by-prettify"> data</span><spa=
n style=3D"color: #660;" class=3D"styled-by-prettify">:</span><span style=
=3D"color: #000;" class=3D"styled-by-prettify"> </span><span style=3D"color=
: #660;" class=3D"styled-by-prettify">({</span><span style=3D"color: #080;"=
 class=3D"styled-by-prettify">&#39;csrf_token&#39;</span><span style=3D"col=
or: #660;" class=3D"styled-by-prettify">:</span><span style=3D"color: #000;=
" class=3D"styled-by-prettify"> </span><span style=3D"color: #080;" class=
=3D"styled-by-prettify">&#39;c8cc04fa-6cec-4dc2-a690-d3d1398f6322&#39;</spa=
n><span style=3D"color: #660;" class=3D"styled-by-prettify">,</span><span s=
tyle=3D"color: #000;" class=3D"styled-by-prettify"> </span><span style=3D"c=
olor: #080;" class=3D"styled-by-prettify">&#39;login&#39;</span><span style=
=3D"color: #660;" class=3D"styled-by-prettify">:</span><span style=3D"color=
: #000;" class=3D"styled-by-prettify"> </span><span style=3D"color: #080;" =
class=3D"styled-by-prettify">&#39;rh&#39;</span><span style=3D"color: #660;=
" class=3D"styled-by-prettify">},</span><span style=3D"color: #000;" class=
=3D"styled-by-prettify"> datetime</span><span style=3D"color: #660;" class=
=3D"styled-by-prettify">.</span><span style=3D"color: #000;" class=3D"style=
d-by-prettify">datetime</span><span style=3D"color: #660;" class=3D"styled-=
by-prettify">(</span><span style=3D"color: #066;" class=3D"styled-by-pretti=
fy">2019</span><span style=3D"color: #660;" class=3D"styled-by-prettify">,<=
/span><span style=3D"color: #000;" class=3D"styled-by-prettify"> </span><sp=
an style=3D"color: #066;" class=3D"styled-by-prettify">5</span><span style=
=3D"color: #660;" class=3D"styled-by-prettify">,</span><span style=3D"color=
: #000;" class=3D"styled-by-prettify"> </span><span style=3D"color: #066;" =
class=3D"styled-by-prettify">10</span><span style=3D"color: #660;" class=3D=
"styled-by-prettify">,</span><span style=3D"color: #000;" class=3D"styled-b=
y-prettify"> </span><span style=3D"color: #066;" class=3D"styled-by-prettif=
y">19</span><span style=3D"color: #660;" class=3D"styled-by-prettify">,</sp=
an><span style=3D"color: #000;" class=3D"styled-by-prettify"> </span><span =
style=3D"color: #066;" class=3D"styled-by-prettify">29</span><span style=3D=
"color: #660;" class=3D"styled-by-prettify">,</span><span style=3D"color: #=
000;" class=3D"styled-by-prettify"> </span><span style=3D"color: #066;" cla=
ss=3D"styled-by-prettify">26</span><span style=3D"color: #660;" class=3D"st=
yled-by-prettify">,</span><span style=3D"color: #000;" class=3D"styled-by-p=
rettify"> </span><span style=3D"color: #066;" class=3D"styled-by-prettify">=
591346</span><span style=3D"color: #660;" class=3D"styled-by-prettify">))</=
span><span style=3D"color: #000;" class=3D"styled-by-prettify"><br>saving s=
ession data</span><span style=3D"color: #660;" class=3D"styled-by-prettify"=
>:</span><span style=3D"color: #000;" class=3D"styled-by-prettify"> dict_it=
ems</span><span style=3D"color: #660;" class=3D"styled-by-prettify">([(</sp=
an><span style=3D"color: #080;" class=3D"styled-by-prettify">&#39;csrf_toke=
n&#39;</span><span style=3D"color: #660;" class=3D"styled-by-prettify">,</s=
pan><span style=3D"color: #000;" class=3D"styled-by-prettify"> </span><span=
 style=3D"color: #080;" class=3D"styled-by-prettify">&#39;c8cc04fa-6cec-4dc=
2-a690-d3d1398f6322&#39;</span><span style=3D"color: #660;" class=3D"styled=
-by-prettify">),</span><span style=3D"color: #000;" class=3D"styled-by-pret=
tify"> </span><span style=3D"color: #660;" class=3D"styled-by-prettify">(</=
span><span style=3D"color: #080;" class=3D"styled-by-prettify">&#39;login&#=
39;</span><span style=3D"color: #660;" class=3D"styled-by-prettify">,</span=
><span style=3D"color: #000;" class=3D"styled-by-prettify"> </span><span st=
yle=3D"color: #080;" class=3D"styled-by-prettify">&#39;rh&#39;</span><span =
style=3D"color: #660;" class=3D"styled-by-prettify">)])</span><span style=
=3D"color: #000;" class=3D"styled-by-prettify"><br></span><span style=3D"co=
lor: #606;" class=3D"styled-by-prettify">UnixLockFile</span><span style=3D"=
color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"color: #0=
00;" class=3D"styled-by-prettify">release called </span><span style=3D"colo=
r: #008;" class=3D"styled-by-prettify">for</span><span style=3D"color: #000=
;" class=3D"styled-by-prettify"> </span><span style=3D"color: #660;" class=
=3D"styled-by-prettify">/</span><span style=3D"color: #000;" class=3D"style=
d-by-prettify">fs</span><span style=3D"color: #660;" class=3D"styled-by-pre=
ttify">/</span><span style=3D"color: #008;" class=3D"styled-by-prettify">lo=
cal</span><span style=3D"color: #660;" class=3D"styled-by-prettify">/</span=
><span style=3D"color: #000;" class=3D"styled-by-prettify">run</span><span =
style=3D"color: #660;" class=3D"styled-by-prettify">/</span><span style=3D"=
color: #000;" class=3D"styled-by-prettify">sessions</span><span style=3D"co=
lor: #660;" class=3D"styled-by-prettify">/</span><span style=3D"color: #000=
;" class=3D"styled-by-prettify">session</span><span style=3D"color: #660;" =
class=3D"styled-by-prettify">-</span><span style=3D"color: #066;" class=3D"=
styled-by-prettify">3146212444067902ec35e41a4af0db64bcbacf4b</span><span st=
yle=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"co=
lor: #008;" class=3D"styled-by-prettify">lock</span><span style=3D"color: #=
000;" class=3D"styled-by-prettify"> </span><span style=3D"color: #660;" cla=
ss=3D"styled-by-prettify">(</span><span style=3D"color: #000;" class=3D"sty=
led-by-prettify">thread</span><span style=3D"color: #660;" class=3D"styled-=
by-prettify">=3D</span><span style=3D"color: #066;" class=3D"styled-by-pret=
tify">139877983168256</span><span style=3D"color: #660;" class=3D"styled-by=
-prettify">)</span><span style=3D"color: #000;" class=3D"styled-by-prettify=
"><br></span><span style=3D"color: #066;" class=3D"styled-by-prettify">2019=
</span><span style=3D"color: #660;" class=3D"styled-by-prettify">-</span><s=
pan style=3D"color: #066;" class=3D"styled-by-prettify">05</span><span styl=
e=3D"color: #660;" class=3D"styled-by-prettify">-</span><span style=3D"colo=
r: #066;" class=3D"styled-by-prettify">10</span><span style=3D"color: #000;=
" class=3D"styled-by-prettify"> </span><span style=3D"color: #066;" class=
=3D"styled-by-prettify">11</span><span style=3D"color: #660;" class=3D"styl=
ed-by-prettify">:</span><span style=3D"color: #066;" class=3D"styled-by-pre=
ttify">29</span><span style=3D"color: #660;" class=3D"styled-by-prettify">:=
</span><span style=3D"color: #066;" class=3D"styled-by-prettify">27</span><=
span style=3D"color: #660;" class=3D"styled-by-prettify">,</span><span styl=
e=3D"color: #066;" class=3D"styled-by-prettify">141</span><span style=3D"co=
lor: #660;" class=3D"styled-by-prettify">:</span><span style=3D"color: #000=
;" class=3D"styled-by-prettify">pid</span><span style=3D"color: #660;" clas=
s=3D"styled-by-prettify">=3D</span><span style=3D"color: #066;" class=3D"st=
yled-by-prettify">22163</span><span style=3D"color: #660;" class=3D"styled-=
by-prettify">:</span><span style=3D"color: #066;" class=3D"styled-by-pretti=
fy">127.0</span><span style=3D"color: #660;" class=3D"styled-by-prettify">.=
</span><span style=3D"color: #066;" class=3D"styled-by-prettify">0.1</span>=
<span style=3D"color: #000;" class=3D"styled-by-prettify"> </span><span sty=
le=3D"color: #660;" class=3D"styled-by-prettify">-</span><span style=3D"col=
or: #000;" class=3D"styled-by-prettify"> rh300070 </span><span style=3D"col=
or: #660;" class=3D"styled-by-prettify">[</span><span style=3D"color: #066;=
" class=3D"styled-by-prettify">10</span><span style=3D"color: #660;" class=
=3D"styled-by-prettify">/</span><span style=3D"color: #606;" class=3D"style=
d-by-prettify">May</span><span style=3D"color: #660;" class=3D"styled-by-pr=
ettify">/</span><span style=3D"color: #066;" class=3D"styled-by-prettify">2=
019</span><span style=3D"color: #660;" class=3D"styled-by-prettify">:</span=
><span style=3D"color: #066;" class=3D"styled-by-prettify">11</span><span s=
tyle=3D"color: #660;" class=3D"styled-by-prettify">:</span><span style=3D"c=
olor: #066;" class=3D"styled-by-prettify">29</span><span style=3D"color: #6=
60;" class=3D"styled-by-prettify">:</span><span style=3D"color: #066;" clas=
s=3D"styled-by-prettify">27</span><span style=3D"color: #660;" class=3D"sty=
led-by-prettify">]</span><span style=3D"color: #000;" class=3D"styled-by-pr=
ettify"> </span><span style=3D"color: #080;" class=3D"styled-by-prettify">&=
quot;GET /CP/getPlot?image_id=3Di_1 HTTP/1.1&quot;</span><span style=3D"col=
or: #000;" class=3D"styled-by-prettify"> </span><span style=3D"color: #066;=
" class=3D"styled-by-prettify">200</span><span style=3D"color: #000;" class=
=3D"styled-by-prettify"> </span><span style=3D"color: #066;" class=3D"style=
d-by-prettify">14708</span><span style=3D"color: #000;" class=3D"styled-by-=
prettify"> </span><span style=3D"color: #080;" class=3D"styled-by-prettify"=
>&quot;http://localhost:8080/CP/static/quant.html&quot;</span><span style=
=3D"color: #000;" class=3D"styled-by-prettify"> <br><br></span><span style=
=3D"color: #606;" class=3D"styled-by-prettify">UnixLockFile</span><span sty=
le=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"col=
or: #000;" class=3D"styled-by-prettify">_lock_file called </span><span styl=
e=3D"color: #008;" class=3D"styled-by-prettify">for</span><span style=3D"co=
lor: #000;" class=3D"styled-by-prettify"> </span><span style=3D"color: #660=
;" class=3D"styled-by-prettify">/</span><span style=3D"color: #000;" class=
=3D"styled-by-prettify">fs</span><span style=3D"color: #660;" class=3D"styl=
ed-by-prettify">/</span><span style=3D"color: #008;" class=3D"styled-by-pre=
ttify">local</span><span style=3D"color: #660;" class=3D"styled-by-prettify=
">/</span><span style=3D"color: #000;" class=3D"styled-by-prettify">run</sp=
an><span style=3D"color: #660;" class=3D"styled-by-prettify">/</span><span =
style=3D"color: #000;" class=3D"styled-by-prettify">sessions</span><span st=
yle=3D"color: #660;" class=3D"styled-by-prettify">/</span><span style=3D"co=
lor: #000;" class=3D"styled-by-prettify">session</span><span style=3D"color=
: #660;" class=3D"styled-by-prettify">-</span><span style=3D"color: #066;" =
class=3D"styled-by-prettify">3146212444067902ec35e41a4af0db64bcbacf4b</span=
><span style=3D"color: #660;" class=3D"styled-by-prettify">.</span><span st=
yle=3D"color: #008;" class=3D"styled-by-prettify">lock</span><span style=3D=
"color: #000;" class=3D"styled-by-prettify"> on fileno </span><span style=
=3D"color: #066;" class=3D"styled-by-prettify">36</span><span style=3D"colo=
r: #000;" class=3D"styled-by-prettify"> </span><span style=3D"color: #660;"=
 class=3D"styled-by-prettify">(</span><span style=3D"color: #000;" class=3D=
"styled-by-prettify">thread</span><span style=3D"color: #660;" class=3D"sty=
led-by-prettify">=3D</span><span style=3D"color: #066;" class=3D"styled-by-=
prettify">139877479868160</span><span style=3D"color: #660;" class=3D"style=
d-by-prettify">)</span><span style=3D"color: #000;" class=3D"styled-by-pret=
tify"><br></span><span style=3D"color: #606;" class=3D"styled-by-prettify">=
Loaded</span><span style=3D"color: #000;" class=3D"styled-by-prettify"> ses=
sion </span><span style=3D"color: #008;" class=3D"styled-by-prettify">with<=
/span><span style=3D"color: #000;" class=3D"styled-by-prettify"> data</span=
><span style=3D"color: #660;" class=3D"styled-by-prettify">:</span><span st=
yle=3D"color: #000;" class=3D"styled-by-prettify"> </span><span style=3D"co=
lor: #660;" class=3D"styled-by-prettify">({</span><span style=3D"color: #08=
0;" class=3D"styled-by-prettify">&#39;csrf_token&#39;</span><span style=3D"=
color: #660;" class=3D"styled-by-prettify">:</span><span style=3D"color: #0=
00;" class=3D"styled-by-prettify"> </span><span style=3D"color: #080;" clas=
s=3D"styled-by-prettify">&#39;c8cc04fa-6cec-4dc2-a690-d3d1398f6322&#39;</sp=
an><span style=3D"color: #660;" class=3D"styled-by-prettify">,</span><span =
style=3D"color: #000;" class=3D"styled-by-prettify"> </span><span style=3D"=
color: #080;" class=3D"styled-by-prettify">&#39;login&#39;</span><span styl=
e=3D"color: #660;" class=3D"styled-by-prettify">:</span><span style=3D"colo=
r: #000;" class=3D"styled-by-prettify"> </span><span style=3D"color: #080;"=
 class=3D"styled-by-prettify">&#39;rh&#39;</span><span style=3D"color: #660=
;" class=3D"styled-by-prettify">},</span><span style=3D"color: #000;" class=
=3D"styled-by-prettify"> datetime</span><span style=3D"color: #660;" class=
=3D"styled-by-prettify">.</span><span style=3D"color: #000;" class=3D"style=
d-by-prettify">datetime</span><span style=3D"color: #660;" class=3D"styled-=
by-prettify">(</span><span style=3D"color: #066;" class=3D"styled-by-pretti=
fy">2019</span><span style=3D"color: #660;" class=3D"styled-by-prettify">,<=
/span><span style=3D"color: #000;" class=3D"styled-by-prettify"> </span><sp=
an style=3D"color: #066;" class=3D"styled-by-prettify">5</span><span style=
=3D"color: #660;" class=3D"styled-by-prettify">,</span><span style=3D"color=
: #000;" class=3D"styled-by-prettify"> </span><span style=3D"color: #066;" =
class=3D"styled-by-prettify">10</span><span style=3D"color: #660;" class=3D=
"styled-by-prettify">,</span><span style=3D"color: #000;" class=3D"styled-b=
y-prettify"> </span><span style=3D"color: #066;" class=3D"styled-by-prettif=
y">19</span><span style=3D"color: #660;" class=3D"styled-by-prettify">,</sp=
an><span style=3D"color: #000;" class=3D"styled-by-prettify"> </span><span =
style=3D"color: #066;" class=3D"styled-by-prettify">29</span><span style=3D=
"color: #660;" class=3D"styled-by-prettify">,</span><span style=3D"color: #=
000;" class=3D"styled-by-prettify"> </span><span style=3D"color: #066;" cla=
ss=3D"styled-by-prettify">27</span><span style=3D"color: #660;" class=3D"st=
yled-by-prettify">,</span><span style=3D"color: #000;" class=3D"styled-by-p=
rettify"> </span><span style=3D"color: #066;" class=3D"styled-by-prettify">=
146440</span><span style=3D"color: #660;" class=3D"styled-by-prettify">))</=
span><span style=3D"color: #000;" class=3D"styled-by-prettify"><br>saving s=
ession data</span><span style=3D"color: #660;" class=3D"styled-by-prettify"=
>:</span><span style=3D"color: #000;" class=3D"styled-by-prettify"> dict_it=
ems</span><span style=3D"color: #660;" class=3D"styled-by-prettify">([(</sp=
an><span style=3D"color: #080;" class=3D"styled-by-prettify">&#39;csrf_toke=
n&#39;</span><span style=3D"color: #660;" class=3D"styled-by-prettify">,</s=
pan><span style=3D"color: #000;" class=3D"styled-by-prettify"> </span><span=
 style=3D"color: #080;" class=3D"styled-by-prettify">&#39;c8cc04fa-6cec-4dc=
2-a690-d3d1398f6322&#39;</span><span style=3D"color: #660;" class=3D"styled=
-by-prettify">),</span><span style=3D"color: #000;" class=3D"styled-by-pret=
tify"> </span><span style=3D"color: #660;" class=3D"styled-by-prettify">(</=
span><span style=3D"color: #080;" class=3D"styled-by-prettify">&#39;login&#=
39;</span><span style=3D"color: #660;" class=3D"styled-by-prettify">,</span=
><span style=3D"color: #000;" class=3D"styled-by-prettify"> </span><span st=
yle=3D"color: #080;" class=3D"styled-by-prettify">&#39;rh&#39;</span><span =
style=3D"color: #660;" class=3D"styled-by-prettify">)])</span><span style=
=3D"color: #000;" class=3D"styled-by-prettify"><br></span><span style=3D"co=
lor: #606;" class=3D"styled-by-prettify">UnixLockFile</span><span style=3D"=
color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"color: #0=
00;" class=3D"styled-by-prettify">release called </span><span style=3D"colo=
r: #008;" class=3D"styled-by-prettify">for</span><span style=3D"color: #000=
;" class=3D"styled-by-prettify"> </span><span style=3D"color: #660;" class=
=3D"styled-by-prettify">/</span><span style=3D"color: #000;" class=3D"style=
d-by-prettify">fs</span><span style=3D"color: #660;" class=3D"styled-by-pre=
ttify">/</span><span style=3D"color: #008;" class=3D"styled-by-prettify">lo=
cal</span><span style=3D"color: #660;" class=3D"styled-by-prettify">/</span=
><span style=3D"color: #000;" class=3D"styled-by-prettify">run</span><span =
style=3D"color: #660;" class=3D"styled-by-prettify">/</span><span style=3D"=
color: #000;" class=3D"styled-by-prettify">sessions</span><span style=3D"co=
lor: #660;" class=3D"styled-by-prettify">/</span><span style=3D"color: #000=
;" class=3D"styled-by-prettify">session</span><span style=3D"color: #660;" =
class=3D"styled-by-prettify">-</span><span style=3D"color: #066;" class=3D"=
styled-by-prettify">3146212444067902ec35e41a4af0db64bcbacf4b</span><span st=
yle=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"co=
lor: #008;" class=3D"styled-by-prettify">lock</span><span style=3D"color: #=
000;" class=3D"styled-by-prettify"> </span><span style=3D"color: #660;" cla=
ss=3D"styled-by-prettify">(</span><span style=3D"color: #000;" class=3D"sty=
led-by-prettify">thread</span><span style=3D"color: #660;" class=3D"styled-=
by-prettify">=3D</span><span style=3D"color: #066;" class=3D"styled-by-pret=
tify">139877479868160</span><span style=3D"color: #660;" class=3D"styled-by=
-prettify">)</span><span style=3D"color: #000;" class=3D"styled-by-prettify=
"><br></span><span style=3D"color: #066;" class=3D"styled-by-prettify">2019=
</span><span style=3D"color: #660;" class=3D"styled-by-prettify">-</span><s=
pan style=3D"color: #066;" class=3D"styled-by-prettify">05</span><span styl=
e=3D"color: #660;" class=3D"styled-by-prettify">-</span><span style=3D"colo=
r: #066;" class=3D"styled-by-prettify">10</span><span style=3D"color: #000;=
" class=3D"styled-by-prettify"> </span><span style=3D"color: #066;" class=
=3D"styled-by-prettify">11</span><span style=3D"color: #660;" class=3D"styl=
ed-by-prettify">:</span><span style=3D"color: #066;" class=3D"styled-by-pre=
ttify">29</span><span style=3D"color: #660;" class=3D"styled-by-prettify">:=
</span><span style=3D"color: #066;" class=3D"styled-by-prettify">27</span><=
span style=3D"color: #660;" class=3D"styled-by-prettify">,</span><span styl=
e=3D"color: #066;" class=3D"styled-by-prettify">153</span><span style=3D"co=
lor: #660;" class=3D"styled-by-prettify">:</span><span style=3D"color: #000=
;" class=3D"styled-by-prettify">pid</span><span style=3D"color: #660;" clas=
s=3D"styled-by-prettify">=3D</span><span style=3D"color: #066;" class=3D"st=
yled-by-prettify">22163</span><span style=3D"color: #660;" class=3D"styled-=
by-prettify">:</span><span style=3D"color: #066;" class=3D"styled-by-pretti=
fy">127.0</span><span style=3D"color: #660;" class=3D"styled-by-prettify">.=
</span><span style=3D"color: #066;" class=3D"styled-by-prettify">0.1</span>=
<span style=3D"color: #000;" class=3D"styled-by-prettify"> </span><span sty=
le=3D"color: #660;" class=3D"styled-by-prettify">-</span><span style=3D"col=
or: #000;" class=3D"styled-by-prettify"> rh300070 </span><span style=3D"col=
or: #660;" class=3D"styled-by-prettify">[</span><span style=3D"color: #066;=
" class=3D"styled-by-prettify">10</span><span style=3D"color: #660;" class=
=3D"styled-by-prettify">/</span><span style=3D"color: #606;" class=3D"style=
d-by-prettify">May</span><span style=3D"color: #660;" class=3D"styled-by-pr=
ettify">/</span><span style=3D"color: #066;" class=3D"styled-by-prettify">2=
019</span><span style=3D"color: #660;" class=3D"styled-by-prettify">:</span=
><span style=3D"color: #066;" class=3D"styled-by-prettify">11</span><span s=
tyle=3D"color: #660;" class=3D"styled-by-prettify">:</span><span style=3D"c=
olor: #066;" class=3D"styled-by-prettify">29</span><span style=3D"color: #6=
60;" class=3D"styled-by-prettify">:</span><span style=3D"color: #066;" clas=
s=3D"styled-by-prettify">27</span><span style=3D"color: #660;" class=3D"sty=
led-by-prettify">]</span><span style=3D"color: #000;" class=3D"styled-by-pr=
ettify"> </span><span style=3D"color: #080;" class=3D"styled-by-prettify">&=
quot;GET /CP/getPlot?image_id=3Di_7 HTTP/1.1&quot;</span><span style=3D"col=
or: #000;" class=3D"styled-by-prettify"> </span><span style=3D"color: #066;=
" class=3D"styled-by-prettify">200</span><span style=3D"color: #000;" class=
=3D"styled-by-prettify"> </span><span style=3D"color: #066;" class=3D"style=
d-by-prettify">14708</span><span style=3D"color: #000;" class=3D"styled-by-=
prettify"> </span><span style=3D"color: #080;" class=3D"styled-by-prettify"=
>&quot;http://localhost:8080/CP/static/quant.html&quot;</span><span style=
=3D"color: #000;" class=3D"styled-by-prettify"><br><br></span><span style=
=3D"color: #606;" class=3D"styled-by-prettify">UnixLockFile</span><span sty=
le=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"col=
or: #000;" class=3D"styled-by-prettify">_lock_file called </span><span styl=
e=3D"color: #008;" class=3D"styled-by-prettify">for</span><span style=3D"co=
lor: #000;" class=3D"styled-by-prettify"> </span><span style=3D"color: #660=
;" class=3D"styled-by-prettify">/</span><span style=3D"color: #000;" class=
=3D"styled-by-prettify">fs</span><span style=3D"color: #660;" class=3D"styl=
ed-by-prettify">/</span><span style=3D"color: #008;" class=3D"styled-by-pre=
ttify">local</span><span style=3D"color: #660;" class=3D"styled-by-prettify=
">/</span><span style=3D"color: #000;" class=3D"styled-by-prettify">run</sp=
an><span style=3D"color: #660;" class=3D"styled-by-prettify">/</span><span =
style=3D"color: #000;" class=3D"styled-by-prettify">sessions</span><span st=
yle=3D"color: #660;" class=3D"styled-by-prettify">/</span><span style=3D"co=
lor: #000;" class=3D"styled-by-prettify">session</span><span style=3D"color=
: #660;" class=3D"styled-by-prettify">-</span><span style=3D"color: #066;" =
class=3D"styled-by-prettify">3146212444067902ec35e41a4af0db64bcbacf4b</span=
><span style=3D"color: #660;" class=3D"styled-by-prettify">.</span><span st=
yle=3D"color: #008;" class=3D"styled-by-prettify">lock</span><span style=3D=
"color: #000;" class=3D"styled-by-prettify"> on fileno </span><span style=
=3D"color: #066;" class=3D"styled-by-prettify">32</span><span style=3D"colo=
r: #000;" class=3D"styled-by-prettify"> </span><span style=3D"color: #660;"=
 class=3D"styled-by-prettify">(</span><span style=3D"color: #000;" class=3D=
"styled-by-prettify">thread</span><span style=3D"color: #660;" class=3D"sty=
led-by-prettify">=3D</span><span style=3D"color: #066;" class=3D"styled-by-=
prettify">139877454690048</span><span style=3D"color: #660;" class=3D"style=
d-by-prettify">)</span><span style=3D"color: #000;" class=3D"styled-by-pret=
tify"><br></span><span style=3D"color: #606;" class=3D"styled-by-prettify">=
UnixLockFile</span><span style=3D"color: #660;" class=3D"styled-by-prettify=
">.</span><span style=3D"color: #000;" class=3D"styled-by-prettify">_lock_f=
ile called </span><span style=3D"color: #008;" class=3D"styled-by-prettify"=
>for</span><span style=3D"color: #000;" class=3D"styled-by-prettify"> </spa=
n><span style=3D"color: #660;" class=3D"styled-by-prettify">/</span><span s=
tyle=3D"color: #000;" class=3D"styled-by-prettify">fs</span><span style=3D"=
color: #660;" class=3D"styled-by-prettify">/</span><span style=3D"color: #0=
08;" class=3D"styled-by-prettify">local</span><span style=3D"color: #660;" =
class=3D"styled-by-prettify">/</span><span style=3D"color: #000;" class=3D"=
styled-by-prettify">run</span><span style=3D"color: #660;" class=3D"styled-=
by-prettify">/</span><span style=3D"color: #000;" class=3D"styled-by-pretti=
fy">sessions</span><span style=3D"color: #660;" class=3D"styled-by-prettify=
">/</span><span style=3D"color: #000;" class=3D"styled-by-prettify">session=
</span><span style=3D"color: #660;" class=3D"styled-by-prettify">-</span><s=
pan style=3D"color: #066;" class=3D"styled-by-prettify">3146212444067902ec3=
5e41a4af0db64bcbacf4b</span><span style=3D"color: #660;" class=3D"styled-by=
-prettify">.</span><span style=3D"color: #008;" class=3D"styled-by-prettify=
">lock</span><span style=3D"color: #000;" class=3D"styled-by-prettify"> on =
fileno </span><span style=3D"color: #066;" class=3D"styled-by-prettify">36<=
/span><span style=3D"color: #000;" class=3D"styled-by-prettify"> </span><sp=
an style=3D"color: #660;" class=3D"styled-by-prettify">(</span><span style=
=3D"color: #000;" class=3D"styled-by-prettify">thread</span><span style=3D"=
color: #660;" class=3D"styled-by-prettify">=3D</span><span style=3D"color: =
#066;" class=3D"styled-by-prettify">139877479868160</span><span style=3D"co=
lor: #660;" class=3D"styled-by-prettify">)</span><span style=3D"color: #000=
;" class=3D"styled-by-prettify"><br></span><span style=3D"color: #660;" cla=
ss=3D"styled-by-prettify">[...]</span><span style=3D"color: #000;" class=3D=
"styled-by-prettify"><br></span></div></code></div><br>The first two reques=
ts looked entirely ok, the lock was acquired, the session was loaded, then =
after the request the session got saved, and the lock released. <br><br>But=
 then suddenly two threads obtain a lock to the same file! How is this poss=
ible? I have to mention that this does not happen every time I load the mai=
n page, sometimes I have to reload 30 or 40 times until this behavior shows=
 up.<br><br>Looking at the code I have no idea how this can happen, the cod=
e (it uses <code class=3D"descclassname">fcntl.</code><code class=3D"descna=
me">flock()<span style=3D"font-family: arial,sans-serif;"> with correct fla=
gs) looks entirely OK to me. <br><br>Any help is very much appreciated.<br>=
<br>PS: I work under Centos7, Python 3.6.4, cherrypy ~=3D15.0.0 (but also t=
he latest 18.x.x version of cherrypy shows the same behavior)</span><br></c=
ode></div>

<p></p>

-- <br />
You received this message because you are subscribed to the Google Groups &=
quot;cherrypy-users&quot; group.<br />
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:cherrypy-users+unsubscribe-/[email protected]">cher=
rypy-users+unsubscribe-/[email protected]</a>.<br />
To post to this group, send email to <a href=3D"mailto:cherrypy-users@googl=
egroups.com">cherrypy-users-/[email protected]</a>.<br />
Visit this group at <a href=3D"https://groups.google.com/group/cherrypy-use=
rs">https://groups.google.com/group/cherrypy-users</a>.<br />
To view this discussion on the web visit <a href=3D"https://groups.google.c=
om/d/msgid/cherrypy-users/5460092d-e7e7-49a3-974d-d3cdc1c05885%40googlegrou=
ps.com?utm_medium=3Demail&utm_source=3Dfooter">https://groups.google.com/d/=
msgid/cherrypy-users/5460092d-e7e7-49a3-974d-d3cdc1c05885%40googlegroups.co=
m</a>.<br />
For more options, visit <a href=3D"https://groups.google.com/d/optout">http=
s://groups.google.com/d/optout</a>.<br />

------=_Part_864_1312001152.1557490126830--

------=_Part_863_848813383.1557490126828--