Re: thread resource leaks if thread terminates with uncaught exception
Shiro Kawai <[email protected]> Thu, 19 Dec 2024 23:05:46 -1000
| Newsgroups | gmane.lisp.scheme.gauche |
|---|---|
| Message-ID | <CALN0JNHejijo34Tvc9wUZe00nwX9bx7jqzsdvW-jEsBxV55bnA@mail.gmail.com> |
--===============4316014246242014820==
Content-Type: multipart/alternative; boundary="000000000000706f320629aff4aa"
--000000000000706f320629aff4aa
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
A clue. If we remove reference to each threads in the variable `threads`
as follows, all VMs are finalized and the next gc closes opened fds.
```
(cond [(not body)
(for-each thread-terminate! threads)
(let loop ((threads threads))
(unless (null? threads)
(set-car! threads #f)
(loop (cdr threads))))
```
Merely nulling the variable `threads` doesn't work. So somebody's grabbing
the head of thread list itself, preventing them from being GC-ed.
On Tue, Dec 17, 2024 at 10:41=E2=80=AFPM Shiro Kawai <[email protected]=
> wrote:
> Confirmed that if I comment out (when (=3D i 100) (error "i=3D100")) line=
, all
> the terminated VMs are finalized in the next GC cycle, and that closes th=
e
> sockets. If there's that error line, finalizers of VMs are never called.
>
> Regarding avoiding thread-terminate! - it's a tough issue. In general we
> don't know whether a resource allocated in a VM should survive the VM or
> not. And we can't do much with the resources implicitly allocated in a
> library function. I feel robust GC is the only way in this case.
>
>
> On Tue, Dec 17, 2024 at 10:06=E2=80=AFPM Jens Thiele <[email protected]> wro=
te:
>
>> Shiro Kawai <[email protected]> writes:
>>
>> > Ok, I reproduced it. The threads leave the socket to the localhost:80=
81
>> > open (You can see it with `lsof -p <pid>`), which consumes the fd poo=
l.
>> > `thread-terminate!` doesn't give the thread to cleanup, so relying on =
it
>> > should generally be avoided.
>>
>> The problem is how to avoid thread-terminate! in such cases. In this
>> example it is the http-get which has no timeout. I also tried a version
>> where the http-get is done in a child process and canceled with a
>> sys-alarm - but this is really slow and sys-alarm only accepts full
>> seconds as timeout. Maybe one could keep some worker processes around
>> and use select but I think this gets complicated.
>>
>> Best regards
>> Jens
>>
>>
>> _______________________________________________
>> Gauche-devel mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/gauche-devel
>>
>
--000000000000706f320629aff4aa
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">A clue.=C2=A0 If we remove reference to each threads in th=
e variable `threads` as follows, all VMs are finalized and the next gc clos=
es opened fds.<div><br></div><div>```<br><div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 (cond [(not body)<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0(for-each thread-terminate! threads)<br></div><div>=C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(let loop ((threads =
threads))<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=
=C2=A0(unless (null? threads)<br>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(set-car! threads #f)<br>=C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0(loop (cdr th=
reads))))<br></div></div><div>```</div><div><br></div><div>Merely nulling t=
he variable `threads` doesn't work.=C2=A0 So somebody's grabbing th=
e head of thread list itself, preventing them from being GC-ed.</div><div><=
br></div><div><br></div><div><br></div><div><br></div></div><br><div class=
=3D"gmail_quote gmail_quote_container"><div dir=3D"ltr" class=3D"gmail_attr=
">On Tue, Dec 17, 2024 at 10:41=E2=80=AFPM Shiro Kawai <<a href=3D"mailt=
o:[email protected]">[email protected]</a>> wrote:<br></div><blo=
ckquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left=
:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr">Confirmed th=
at if I comment out (when (=3D i 100) (error "i=3D100")) line, al=
l the terminated VMs are finalized in the next GC cycle, and that closes th=
e sockets.=C2=A0 If there's that error line, finalizers of VMs are neve=
r=C2=A0called.=C2=A0=C2=A0<div><br></div><div>Regarding avoiding thread-ter=
minate! - it's a tough issue.=C2=A0 In general we don't know whethe=
r=C2=A0a resource=C2=A0allocated in a VM should survive the VM or not.=C2=
=A0 And we can't do much with the resources=C2=A0implicitly allocated i=
n a library function.=C2=A0 I feel robust GC is the only way in this case.<=
/div><div><br></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" c=
lass=3D"gmail_attr">On Tue, Dec 17, 2024 at 10:06=E2=80=AFPM Jens Thiele &l=
t;<a href=3D"mailto:[email protected]" target=3D"_blank">[email protected]</a>>=
; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px=
0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Shiro K=
awai <<a href=3D"mailto:[email protected]" target=3D"_blank">shiro.k=
[email protected]</a>> writes:<br>
<br>
> Ok, I reproduced it.=C2=A0 The threads leave the socket to the localho=
st:8081<br>
> open=C2=A0 (You can see it with `lsof -p <pid>`), which consumes=
the fd pool.<br>
> `thread-terminate!` doesn't give the thread to cleanup, so relying=
on it<br>
> should generally be avoided.<br>
<br>
The problem is how to avoid thread-terminate! in such cases. In this<br>
example it is the http-get which has no timeout. I also tried a version<br>
where the http-get is done in a child process and canceled with a<br>
sys-alarm - but this is really slow and sys-alarm only accepts full<br>
seconds as timeout. Maybe one could keep some worker processes around<br>
and use select but I think this gets complicated.<br>
<br>
Best regards<br>
Jens<br>
<br>
<br>
_______________________________________________<br>
Gauche-devel mailing list<br>
<a href=3D"mailto:[email protected]" target=3D"_blank">Gau=
[email protected]</a><br>
<a href=3D"https://lists.sourceforge.net/lists/listinfo/gauche-devel" rel=
=3D"noreferrer" target=3D"_blank">https://lists.sourceforge.net/lists/listi=
nfo/gauche-devel</a><br>
</blockquote></div>
</blockquote></div>
--000000000000706f320629aff4aa--
--===============4316014246242014820==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
--===============4316014246242014820==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
Gauche-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/gauche-devel
--===============4316014246242014820==--