Re: Class finalizer that calls C

Hello,

Just for information, even if better can be done, I finally ended with
managing unreferenced memory with a sb-concurrency:queue that I
populate through sb-ext:finalize and handle in the main process to
delete necessary stuff in Julia. That works like a charm.

All the best,

- Greg

Le ven. 23 févr. 2024 à 11:11, Grégory Vanuxem <[email protected]> a écrit :
>
> First, thanks to Phoe and you, for taking the time to respond.
>
> But to respond to you, it's not possible as far as I know, the
> dictionaries are on the Julia side. Working with "interpreter based"
> applications from another application is never easy I find. For a
> quick glance, here is the Julia interface:
> https://docs.julialang.org/en/v1/manual/embedding/#Embedding-Julia
>
> Basically, you have two options, you "import", at startup for example,
> the functions you need and use it via jl_call[1-*](thefunction, *args)
> or you use a generic string evaluation that will be parsed and
> interpreted. The two will return a generic value that you'll unbox to
> fit C need.
>
> Yesterday I tried to use the 'delete!' function but I need to call it
> via jl_call1(del, "123456789") (1 argument). I also tried via string
> parsing.
>
> For now, I continue what I was doing with another CL implementation,
> but I continue to think about that, I really want to use SBCL. In this
> regard I have a question, on SBCL, threads are somewhat similar to
> generic threads, right? What I mean here is, different threads, here
> the finalizers and the main process, share the same address space?
> What I have in mind right now, for example, that should be doable, is
> to use a global/shared pool, FIFO why not, and let the finalizer
> having write access to add references to no longer referenced
> variables and the main process using it via a scheduler or whatever to
> manage it and delete the unnecessary entries to let Julia retrieve
> allocated memory. It may be silly, I have no fixed idea for now.
>
> Regards,
>
> - Greg
>
>
> Le ven. 23 févr. 2024 à 09:44, Stas Boukarev <[email protected]> a écrit :
> >
> > Don't call jl functions, erase the reference to your object and let the julia gc delete it?
> >
> > On Fri, Feb 23, 2024 at 4:38 AM Grégory Vanuxem <[email protected]> wrote:
> >>
> >> The problem is apparently more complex than I thought. The finalizer
> >> runs in another thread, right?
> >>
> >> And from the Julia documentation:
> >> ===============================================
> >> Thread-safety
> >>
> >> In general, the Julia C API is not fully thread-safe. When embedding
> >> Julia in a multi-threaded application care needs to be taken not to
> >> violate the following restrictions:
> >>
> >> jl_init() may only be called once in the application life-time. The
> >> same applies to jl_atexit_hook(), and it may only be called after
> >> jl_init().
> >> jl_...() API functions may only be called from the thread in which
> >> jl_init() was called, or from threads started by the Julia runtime.
> >> Calling Julia API functions from user-started threads is not
> >> supported, and may lead to undefined behaviour and crashes.
> >>
> >> The second condition above implies that you can not safely call
> >> jl_...() functions from threads that were not started by Julia (the
> >> thread calling jl_init() being the exception). For example, the
> >> following is not supported and will most likely segfault:
> >> ================================================
> >>
> >> So other work needs to be done.
> >>
> >> - Greg
> >>
> >> Le jeu. 22 févr. 2024 à 15:05, Grégory Vanuxem <[email protected]> a écrit :
> >> >
> >> > Hello,
> >> >
> >> > I have a problem with the SBCL finalizer. I do not master it at all.
> >> >
> >> > I explain, I have a very simple class to keep a "trace" of variables
> >> > allocated in another application, here Julia. Its definition:
> >> >
> >> > (defclass jlref ()
> >> >     ((id  :accessor jlrefId   :initarg :id)
> >> >     (val  :accessor jlrefVal  :initarg :val)
> >> >     (type :reader   jlrefType :initarg :type))
> >> >     (:default-initargs :id nil :type nil :val nil))
> >> >
> >> > When _my_ variable (on SBCL) is no longer referenced, what I need is
> >> > to let Julia, on its side, garbage collect its memory allocation when
> >> > necessary. So I want to use the SBCL GC finalizer mechanism. But bad
> >> > things happen. I have spent some time trying to figure out why I
> >> > always encounter issues, SBCL's messages are somehow cryptic to me
> >> > sometimes. I tried different ways to let Julia reclaims its memory
> >> > allocation without success. So, no shame, I ask here.
> >> >
> >> > To create an instance I use the following code (I am a bad CL coder I think):
> >> >
> >> > ==============================================
> >> > (defun |make_jlref| (str type)
> >> >     (let* ((hash (write-to-string (random most-positive-fixnum))) ; base 16?
> >> >             (id (|jl_setindex_wrap_eval_string| 0 hash str))
> >> >             (val (|jl_getindex_wrapped_hash| 0 hash))
> >> >             (ret (make-instance 'jlref :type type :id id :val val)))
> >> >         #+:lispworks (flag-special-free-action hash)
> >> >         #+:cmu (ext:finalize ret (lambda () (free_jlref hash)))
> >> >         #+:sbcl (sb-ext:finalize ret (lambda ()
> >> >                     (sb-alien:alien-funcall
> >> >                         (sb-alien::extern-alien "jl_delete_wrapped_hash"
> >> >                             (sb-alien::function sb-alien::void
> >> >                                 (sb-alien::integer)
> >> >                                 (sb-alien::c-string))) 0 hash)))
> >> >         #+:allegro (excl:schedule-finalization ret (lambda ()
> >> > (free_jlref hash)))
> >> >         ret))
> >> >
> >> > (defun free_jlref (hash)
> >> >     (progn
> >> >         (format t "freeing... ~x~%" hash)
> >> > ;        #+:sbcl (sb-ext:finalize ret (lambda ()
> >> >  ;                   (sb-alien:alien-funcall
> >> >   ;                      (sb-alien::extern-alien "jl_delete_wrapped_hash"
> >> >    ;                         (sb-alien::function sb-alien::void
> >> >     ;                            (sb-alien::integer)
> >> >      ;                           (sb-alien::c-string))) 0 hash)))
> >> >         ; Dirty hack but for an unknown reason the following does not work.
> >> >         (|jl_delete_wrapped_hash| 0 hash)))
> >> >         ;jl_eval_string((concatenate 'string "delete!(refs,\"" hash "\")"))))
> >> > =================================================
> >> >
> >> > Basically, I create an object on the Julia side and store it in a
> >> > dictionary to prevent the Julia garbage collector from acting on it.
> >> >
> >> > So now here come my problems:
> >> > If I use the "actual" code, here is what I obtain:
> >> > ===============================================
> >> > #<SB-SYS:MEMORY-FAULT-ERROR {1001D888F3}>
> >> > CORRUPTION WARNING in SBCL pid 27916 tid 27917:
> >> > Memory fault at 0x10 (pc=0x7f8c85e2d5ed, fp=0x7f8c86c0e760,
> >> > sp=0x7f8c86c0e740) tid 27917
> >> > The integrity of this image is possibly compromised.
> >> > Continuing with fingers crossed.
> >> > WARNING:
> >> >    Error calling finalizer #<FUNCTION (LAMBDA ()
> >> >                                         :IN
> >> >                                         |make_jlref|) {10021B333B}>:
> >> >
> >> > WARNING:
> >> >    Error calling finalizer #<FUNCTION (LAMBDA () :IN |make_jlref|) {54F7DC8B}>:
> >> >   #<SB-INT:COMPILED-PROGRAM-ERROR {1001D97EF3}>
> >> > ==========================================
> >> > etc.
> >> >
> >> > In the previous code, below you can see I have two other solutions,
> >> > but I guess I am coding wrong so the error is, for example:
> >> >
> >> > WARNING:
> >> >    Error calling finalizer #<FUNCTION (LAMBDA () :IN |make_jlref|) {54F7DC6B}>:
> >> >   #<SB-INT:COMPILED-PROGRAM-ERROR {1001DA1473}>
> >> >
> >> >
> >> > And If I load the code (instead of compiling it), the memory is
> >> > retrieved on the SBCL side but the finalizer whatever it is (the three
> >> > solutions) is never called apparently.
> >> >
> >> > Don't know if I am clear.
> >> >
> >> > Regards,
> >> >
> >> > - Greg
> >> >
> >> > PS: of course I force the GC to do a full GC for testing purposes.
> >>
> >>
> >> _______________________________________________
> >> Sbcl-help mailing list
> >> [email protected]
> >> https://lists.sourceforge.net/lists/listinfo/sbcl-help


_______________________________________________
Sbcl-help mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/sbcl-help