[emacs-w3m:13519] Re: lexical-binding question

In [emacs-w3m:13517]
On Tue, 06 Aug 2019 10:47:56 +0200, Michael Heerdegen wrote:
> Katsumi Yamaoka <[email protected]> writes:

>> The following forms are utmost simplified model of the functions
>> that the above form uses:

--8<---------------cut here---------------start------------->8---
;; -*- lexical-binding: t -*-
(defun fn1 () ;; simplified `w3m-w3m-expand-arguments'
  (eval '`,charset))

(defun fn2 (charset) ;; simplified `w3m-rendering-half-dump'
  (ignore charset) ;; Silence the byte compiler.
  (fn1))

(defun fn3 (&optional charset) ;; simplified `w3m-rendering-buffer'
  (fn2 charset))
--8<---------------cut here---------------end--------------->8---

> CHARSET is a lexical variable in fn2 and fn3.  Its scope is limited to
> (textual) references in those functions.  The lexical binding is not
> visible in fn1 no matter what eval '`, tricks you try, because it's out
> of scope.  It would only work as you expect with a dynamically bound
> variable.

Yes, I believe that's absolutely true.  But my wonder is why the
current w3m.el code works.  Please try:

・Eval: (setq w3m-input-coding-system 'binary)
・Perform `edebug-defun' on the function `w3m-w3m-expand-arguments'.
・Run M-x w3m for visiting some web page.

Then, you will see '`,charset and `,charset, that the constant
`w3m-halfdump-command-arguments' defines, are substituted with
a certain charset value successfully, even though the value of
`charset' is not passed to `w3m-w3m-expand-arguments' explicitly.

While edebugging `w3m-w3m-expand-arguments', it is notable that
the form (boundp 'charset) returns t and (symbol-value 'charset)
returns a certain charset value.  Does it mean that `charset' is
not a lexical variable?  If it is true, why it happens, and why
it does not happen with the simplified forms (i.e., fn1~fn3)?

Before turning on lexical-binding, the w3m.el code was like this:

--8<---------------cut here---------------start------------->8---
;; -*- lexical-binding: nil -*-
(defun fn4 () ;; simplified `w3m-w3m-expand-arguments'
  (eval 'charset))

(defun fn5 (charset) ;; simplified `w3m-rendering-half-dump'
  (fn4))

(defun fn6 (&optional charset) ;; simplified `w3m-rendering-buffer'
  (fn5 charset))
--8<---------------cut here---------------end--------------->8---

(fn6 'big5) returns big5.  But it got to cause the void variable
error after turning on lexical-binding in w3m.el.  First I tried
was changing   charset   with   `,charset   in the value form of
the `w3m-halfdump-command-arguments' constant, and saw it made it
work.  Please don't ask me why I did so, it was just at random.

> You can either make `charset' special (that can even be done locally in
> the function) to reuse the old code, or you rewrite that stuff so that
> it's really lexical binding code.  Then w3m-halfdump-command-arguments
> would probably become a function `w3m-halfdump-get-command-arguments'
> that depends on an CHARSET argument, or something similar.

I think changing the constant to a function is TRT, too.  But I
don't want to do it before clearing this charset mystery. ;)

Thanks.