Re: should ccl allow ffi callbacks to be defined within closures

--Apple-Mail-2F0C6F16-0D8D-4B5C-9173-002DDB82DFB1
Content-Type: text/plain;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable

Hi Madhu,

If I may pose a rhetorical question: what is your operational goal for defin=
ing a FFI callback that closes over its lexical environment? Is there perhap=
s a way you can rewrite your code to avoid the need for a closure?

Speaking for myself, in general, I am okay with expecting top-level defining=
 forms to only be allowed at top-level and not inside arbitrary Lisp express=
ions. For example, in my professional Lisp work, I have never once needed (d=
efun foo (x) (defun bar (y) (+ x y))) to work.  And I it wouldn=E2=80=99t oc=
cur to me to write expressions like (let ((mumble (defvar *quux* 42))) (zero=
p mumble))

It is valuable to Lisp implementers that these sorts of top-level expression=
s rules and conventions go unchallenged by users (programmers) so the implem=
enters can write stable and efficient binary loaders and dumpers (the =E2=80=
=9Cfasloader=E2=80=9D inside load and the =E2=80=9Cfasdumper=E2=80=9D inside=
 compile-file) and, in this case, CCL=E2=80=98s already powerful FFI.=20

One way to think about the legality of this example is to ask yourself: what=
 type of value does ccl:defcallback return?=20

The fasdumper can only write certain types of constants to a binary file. Th=
is might explain the error message you received.=20

I apologize if this message seems adversarial or too much like a lecture. I a=
m only trying to offer insights into how a Lisp programmer such as myself na=
vigates through the language=E2=80=99s restrictions by =E2=80=9Cthinking ins=
ide the box.=E2=80=9D Yes, sometimes the =E2=80=9Cbox=E2=80=9D is not the ea=
siest to =E2=80=9Cwrap one=E2=80=99s head around.=E2=80=9D Initially, it too=
k me years to absorb the conventions and unwritten rules.=20

The good news is that Lisp offers tremendous opportunity to think outside th=
e =E2=80=9Cboxes=E2=80=9D imposed by other programming languages. It makes m=
e smile every day.=20

--Tim

> On May 22, 2024, at 02:55, Madhu <[email protected]> wrote:
>=20
> =EF=BB=BFI came across a use of cffi callbacks which defines the callback
> within a function. In CCL the following
>=20
> ```
> (defun barf ()
>  (ccl:defcallback CMPSTRINGP (:address p1 :address p2 :signed-int)
>      (ccl:external-call "strcmp" :address p1 :address p2 :signed-int)))
> (barf)
>=20
> =3D>
> Condition of type TYPE-ERROR:
>=20
> The value (:INTERNAL CMPSTRINGP BARF) is not of the expected type
> (AND SYMBOL (NOT (SATISFIES CONSTANTP)))
> ```
>=20
> Is there any reason to disallow callbacks defined by defcallbacks from
> being closures which capture their environment?
>=20
> I am attaching a patch which continues to use the name specified by
> DEFCALLBACK even if the defining function is a closure.
>=20
> I'd very much appreciate it if other developers could review it and
> give feedback.
>=20
>=20
> =46rom d6241c160cc42979da2629862f97c350615850a9 Mon Sep 17 00:00:00 2001 From:=
 Madhu Date: Sun, 12 May 2024 22:01:08 +0530 Subject: [PATCH] allow ffi call=
backs to be defined within closures. * level-1/l1-callbacks.lisp: (define-ca=
llback-function): If the function-name is an internal-function of the form (=
:internal name) use the internal name instead. After this patch, defcallback=
s can be defined within internal functions. e.g. the following will work, in=
stead of erroring out on the name. ``` (defun barf () (ccl:defcallback CMPST=
RINGP (:address p1 :address p2 :signed-int) (ccl:external-call "strcmp" :add=
ress p1 :address p2 :signed-int))) (barf) ``` --- level-1/l1-callbacks.lisp |=
 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/level-1=
/l1-callbacks.lisp b/level-1/l1-callbacks.lisp index 3f88041..f147f71 100644=
 --- a/level-1/l1-callbacks.lisp +++ b/level-1/l1-callbacks.lisp @@ -25,7 +2=
5,10 @@ (defstatic *callback-lock* (make-lock)) (defun define-callback-funct=
ion (lisp-function &optional doc-string (without-interrupts t) info &aux nam=
e trampoline) (unless (functionp lisp-function) (setq lisp-function (require=
-type lisp-function 'function))) - (unless (and (symbolp (setq name (functio=
n-name lisp-function))) + (if (and (consp (setq name (function-name lisp-fun=
ction))) + (eql (car name) :internal)) + (setq name (second name))) + (unles=
s (and (symbolp name) ;;Might as well err out now before do any _Newptr's...=
 (not (constant-symbol-p name))) (report-bad-arg name '(and symbol (not (sat=
isfies constantp))))) -- 2.39.2.101.g768bb238c4

--Apple-Mail-2F0C6F16-0D8D-4B5C-9173-002DDB82DFB1
Content-Type: text/html;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable

<html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D=
utf-8"></head><body dir=3D"auto">Hi Madhu,<div><br></div><div>If I may pose a=
 rhetorical question: what is your operational goal for defining a FFI callb=
ack that closes over its lexical environment? Is there perhaps a way you can=
 rewrite your code to avoid the need for a closure?<div><br></div><div>Speak=
ing for myself, in general, I am okay with expecting top-level <i>defining f=
orms</i> to only be allowed at top-level and not inside arbitrary Lisp expre=
ssions. For example, in my professional Lisp work, I have never once needed&=
nbsp;<b>(defun foo (x) (defun bar (y) (+ x y)))</b> to work. &nbsp;And I it w=
ouldn=E2=80=99t occur to me to write expressions like <b>(let ((mumble (defv=
ar *quux* 42))) (zerop mumble))</b></div><div><div><br></div><div>It is valu=
able to Lisp <i>implementers</i> that these sorts of top-level expressions r=
ules and conventions go unchallenged by users (programmers) so the implement=
ers can write stable and efficient binary loaders and dumpers (the =E2=80=9C=
fasloader=E2=80=9D inside&nbsp;<b>load</b> and the =E2=80=9Cfasdumper=E2=80=9D=
 inside&nbsp;<b>compile-file</b>) and, in this case, CCL=E2=80=98s already p=
owerful FFI.&nbsp;</div><div><br></div><div>One way to think about the legal=
ity of this example is to ask yourself: what type of value does <b>ccl:defca=
llback</b> return?&nbsp;</div><div><br></div><div>The fasdumper can only wri=
te certain types of constants to a binary file. This might explain the error=
 message you received.&nbsp;</div><div><br></div><div>I apologize if this me=
ssage seems adversarial or too much like a lecture. I am only trying to offe=
r insights into how a Lisp programmer such as myself navigates through the l=
anguage=E2=80=99s restrictions by =E2=80=9Cthinking inside the box.=E2=80=9D=
 Yes, sometimes the =E2=80=9Cbox=E2=80=9D is not the easiest to =E2=80=9Cwra=
p one=E2=80=99s head around.=E2=80=9D Initially, it took me years to absorb t=
he conventions and unwritten rules.&nbsp;</div><div><br></div><div>The good n=
ews is that Lisp offers tremendous opportunity to think outside the =E2=80=9C=
boxes=E2=80=9D imposed by <i>other</i> programming languages. It makes me sm=
ile every day.&nbsp;</div><div><br id=3D"lineBreakAtBeginningOfSignature"><d=
iv dir=3D"ltr">--Tim</div><div dir=3D"ltr"><br><blockquote type=3D"cite">On M=
ay 22, 2024, at 02:55, Madhu &lt;[email protected]&gt; wrote:<br><br></blockq=
uote></div><blockquote type=3D"cite"><div dir=3D"ltr">=EF=BB=BF<span>I came a=
cross a use of cffi callbacks which defines the callback</span><br><span>wit=
hin a function. In CCL the following</span><br><span></span><br><span>```</s=
pan><br><span>(defun barf ()</span><br><span> &nbsp;(ccl:defcallback CMPSTRI=
NGP (:address p1 :address p2 :signed-int)</span><br><span> &nbsp;&nbsp;&nbsp=
;&nbsp;&nbsp;(ccl:external-call "strcmp" :address p1 :address p2 :signed-int=
)))</span><br><span>(barf)</span><br><span></span><br><span>=3D&gt;</span><b=
r><span>Condition of type TYPE-ERROR:</span><br><span></span><br><span>The v=
alue (:INTERNAL CMPSTRINGP BARF) is not of the expected type</span><br><span=
> (AND SYMBOL (NOT (SATISFIES CONSTANTP)))</span><br><span>```</span><br><sp=
an></span><br><span>Is there any reason to disallow callbacks defined by def=
callbacks from</span><br><span>being closures which capture their environmen=
t?</span><br><span></span><br><span>I am attaching a patch which continues t=
o use the name specified by</span><br><span>DEFCALLBACK even if the defining=
 function is a closure.</span><br><span></span><br><span>I'd very much appre=
ciate it if other developers could review it and</span><br><span>give feedba=
ck.</span><br><span></span><br><span></span><br>=46rom d6241c160cc42979da262=
9862f97c350615850a9 Mon Sep 17 00:00:00 2001
From: Madhu <[email protected]>
Date: Sun, 12 May 2024 22:01:08 +0530
Subject: [PATCH] allow ffi callbacks to be defined within closures.

* level-1/l1-callbacks.lisp: (define-callback-function): If the
function-name is an internal-function of the form (:internal name) use
the internal name instead.

After this patch, defcallbacks can be defined within internal
functions. e.g. the following will work, instead of erroring out on
the name.

```
(defun barf ()
  (ccl:defcallback CMPSTRINGP (:address p1 :address p2 :signed-int)
      (ccl:external-call "strcmp" :address p1 :address p2 :signed-int)))
(barf)
```
---
 level-1/l1-callbacks.lisp | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/level-1/l1-callbacks.lisp b/level-1/l1-callbacks.lisp
index 3f88041..f147f71 100644
--- a/level-1/l1-callbacks.lisp
+++ b/level-1/l1-callbacks.lisp
@@ -25,7 +25,10 @@ (defstatic *callback-lock* (make-lock))
 (defun define-callback-function (lisp-function  &amp;optional doc-string (w=
ithout-interrupts t) info &amp;aux name trampoline)
   (unless (functionp lisp-function)
     (setq lisp-function (require-type lisp-function 'function)))
-  (unless (and (symbolp (setq name (function-name lisp-function)))
+  (if (and (consp (setq name (function-name lisp-function)))
+	   (eql (car name) :internal))
+      (setq name (second name)))
+  (unless (and (symbolp name)
                ;;Might as well err out now before do any _Newptr's...
                (not (constant-symbol-p name)))
     (report-bad-arg name '(and symbol (not (satisfies constantp)))))
--=20
2.39.2.101.g768bb238c4

</[email protected]></div></blockquote></div></div></div></body></html>=

--Apple-Mail-2F0C6F16-0D8D-4B5C-9173-002DDB82DFB1--