Re: thread resource leaks if thread terminates with uncaught exception

Jens Thiele <[email protected]> writes:

> Jens Thiele <[email protected]> writes:
>
>> Hi,
>>
>> I have some resource leaks problems with a long running process.
>>
>> I still don't have a really simple test but I am getting closer to the
>> problem.
>>
>> Let's say we have some webserver where some requests are really
>> slow. For example using makiki: server.scm:
>>
>> #!/bin/sh
>> #| -*- mode: scheme; coding: utf-8; -*-
>> exec gosh -I. -- $0 "$@"
>> |#
>> (use makiki)
>> (use text.html-lite)
>>
>> (define (hello)
>>   (list (html-doctype)
>> 	(html:html
>> 	 (html:head (html:title "hello"))
>> 	 (html:body (html:p "hello")))))
>>
>> (define-http-handler (GET)  "/"     (^[req app] (respond/ok req (hello))))
>> (define-http-handler (GET)  "/slow" (^[req app] (sys-sleep 10) (respond/ok req (hello))))
>>
>> (define (main args)
>>   (start-http-server :port 8081)
>>   0)
>>
>> Then we have some reverse proxy also using makiki (that will be the
>> process with the leaks). It tries to fetch many urls with a timeout in
>> parallel and for some reason oaccasionally there might be an error (the
>> "(when (= i 100) (error "i=100"))").
>>
>> #!/bin/sh
>> #| -*- mode: scheme; coding: utf-8; -*-
>> #export GC_PRINT_STATS=1
>> exec gosh -I. -- $0 "$@"
>> |#
>> (use makiki)
>> (use text.html-lite)
>> (use rfc.http)
>> (use gauche.threads)
>> (use file.util)
>> (use runtime-compile)
>>
>> (compile-and-load
>>  `((inline-stub
>>     (define-cproc get-gc-no ()
>>       (let* ((r::(struct GC_prof_stats_s)))
>>         (GC_get_prof_stats (& r) (sizeof r))
>>         (return (SCM_MAKE_INT (ref r gc_no)))))))
>>  '(get-gc-no))
>>
>> ;; todo: linux specific
>> (define (num-open-files) (length (directory-list "/proc/self/fd" :children? #t)))
>>
>> (define-http-handler (GET) "/" (^[req app]
>> 				 (receive (status headers body)
>> 				     (http-get "localhost:8081" "/")
>> 				   (respond/ok req body))))
>>
>> (define-http-handler (GET) "/timeout" (^[req app]
>> 					(let1 threads (map (lambda(i)
>> 							     (make-thread
>> 							      (lambda()
>> 								;; if i comment this one it works
>> 								(when (= i 100) (error "i=100"))
>> 								(receive (status headers body)
>> 								    (http-get "localhost:8081" "/slow")
>> 								  body))))
>> 							   (iota 500))
>
> if I use "only" 50 here - it looks like there is no problem

after quite some time I also got the error with 50 threads. Looks like
it just takes much longer.

Interestingly the first error was again a
"getaddrinfo failed: System error: Device or resource busy"
in one of the threads
I am not sure how this error could happen with a lookup for "localhost"
at all.

Afterwards the main thread exited with
"SYSTEM-ERROR: accept(2) failed: Too many open files"

Best regards
Jens