Re: Lisp Reader & Tools
"Pascal Bourguignon (as informatimago at gmail dot com)" <[email protected]> Thu, 28 May 2026 21:38:32 +0200
| Newsgroups | gmane.lisp.lispworks.general |
|---|---|
| Message-ID | <[email protected]> |
Yes, emacs has several parsers and parser generators to be able to perform sensible syntactic editing. https://www.gnu.org/software/emacs/manual/html_mono/bovine.html https://www.gnu.org/software/emacs/manual/html_node/semantic/index.html -- __Pascal Bourguignon__ [email protected] > On 28 May 2026, at 19:55, David McClain (as dbm at refined-audiometrics dot com) <[email protected]> wrote: > > It would actually be pretty neat if there were some kind of flag symbol to the Editor that says: turn off Lisp mode in this region. Then it could handle the here-docs without getting all confused. > > But the real answer is probably to make a text editor that pays attention to parse trees, and not just text chars. I don’t know if such a thing has ever been written. > >> On May 28, 2026, at 10:47, David McClain (as dbm at refined-audiometrics dot com) <[email protected]> wrote: >> >> Well, yes, technically true. But Lisp sort of ignores lexical analysis conventions by looking *only* for a set of global terminating chars, which are actually themselves starter characters. >> >> Interestingly, my investigations into LW and SBCL shows that the PRINT-OBJECT is sensitive to the current *READTABLE*. >> >> And once I modify the readtable to have the +/ signs and all the numerical digits be starter symbols (non-terminating macro chars), PRINT-OBJECT started putting backslash escapes on all symbol names that contained a digit or a +/- sign. So to avoid that visual mess, I had to provide an :AROUND method on PRINT-OBJECT (SYMBOL, T) to set the readtable to a copy of a virgin table without all the numerical modifications. >> >> So the attitude of the Lisp Reader then bleed over into the Editor, where it assumes we can parse structure by looking only at regular expressions. But it is well known that RE cannot successfully parse advanced forms of syntax, e.g., simply balancing parens. For that, it takes a higher order automaton (e.g., stack machine). And beyond that are LR, LRn, LALR, etc. etc. etc. >> >> So when the Editor tries to find the start of a definition for immediate execution, it stumbles badly in the midst of my bent syntax inside of here-doc strings. >> >> >> >>> On May 28, 2026, at 10:36, Martin Simmons <[email protected]> wrote: >>> >>> I'm not sure what you mean by based on prefix chars. A lexical analyzer for a >>> language with user-defined tokens needs to know the set of token terminating >>> chars, e.g. otherwise how would it distinguish "a1" (a single token for a >>> variable named "a1") from "a 1" (two tokens, a variable named "a" followed by >>> the number 1). In CL, that set is just configurable via the terminating vs >>> non-terminating flag. >>> >>> -- >>> Martin Simmons >>> LispWorks Ltd >>> http://www.lispworks.com/ >>> >>> >>> >>>>>>>> On Thu, 28 May 2026 09:00:01 -0700, David McClain said: >>>> >>>> Oh wow! Very cool! Thanks for the tips, Martin. I accidentally discovered those two characters for just the purpose originally intended. What a lucky random dart! >>>> >>>> My main puzzlement arises from the use of Terminating vs Non-Terminating macro chars. I have always built lexical analyzers based on prefix chars, not termination chars. I see that the Lisp way works. But it also implies that code that looks cramped like this would pass muster: >>>> >>>> (let((c #\:))(find c"my string has colons :::":from-end t)) >>>> >>>> It does, and I suppose it’s okay, but it violates common sense style. >>>> >>>> BTW… I have never used Perl. Is that de rigueur among programmers of today? (I’m not a programmer). I have used SED and AWK about 40-50 years ago. But Lisp can do everything we need. Why go outside the nice environment into Bash land? >>>> >>>>> On May 28, 2026, at 08:33, Martin Simmons <[email protected]> wrote: >>>>> >>>>> They didn't actually escape notice. #\U+2032 and #\U+2033 are unicode "prime" >>>>> characters, so software doesn't typically use them as quotes. According to >>>>> the authors of Wikipedia, these primes are the correct characters to use for >>>>> arcminutes and arcseconds, so using them is good. >>>>> >>>>> We have reader macros for #\U+2018 (as a backquote), #\U+2019 (as a quote) and >>>>> #\U+201C (as a double-quote, paired with #\U+201D) because some software auto >>>>> converts the ascii backquote, quote and double-quote into them, so it is >>>>> convenient to recognize them for cut and paste. Ironically, it looks your >>>>> email client did this auto conversion in your keyboard sequences below! >>>>> >>>>> -- >>>>> Martin Simmons >>>>> LispWorks Ltd >>>>> http://www.lispworks.com/ >>>>> >>>>> >>>>> >>>>>>>>>> On Wed, 27 May 2026 15:29:47 -0700, David McClain (as dbm at refined-audiometrics dot com) said: >>>>>> >>>>>> Found it!! >>>>>> >>>>>> #\U+2032 (′) and #U+2033 (″). These two seem to have escaped notice by LW. And so they can be used in symbol names and anywhere else you like, without confusing the Listener and Editor. >>>>>> >>>>>> I have them assigned to keyboard sequences ^z-‘ and ^z-“ for easy access. My ^z- chords also map all the Greek alphabet. >>>>>> >>>>>> >>>>>> >>>>>>> On May 27, 2026, at 14:31, David McClain <[email protected]> wrote: >>>>>>> >>>>>>> And also, many times, I just want to copy/paste from a printout from some other source and have Lisp interpret what is shown. Hence the desire to have embedded quote and double quote in the angle input. To use #N/xx…/ means I have to pre-edit the printout before submitting to Lisp. >>>>>>> >>>>>>> But… if we had an extensible Lexical Analyzer that could be tailored to the needs…. >>>>>>> >>>>>>> What Common Lisp presents is cutting edge for 1980’s and earlier. But not so much for today. >>>>>>> >>>>>>>> On May 27, 2026, at 14:21, David McClain (as dbm at refined-audiometrics dot com) <[email protected]> wrote: >>>>>>>> >>>>>>>> …and also because the Editor and Listener balks when your delimited run of chars contains an isolated or odd number of |, “, or an embedded \ that isn’t being used as an escape. This speaks to the need for those bizarre comments that have to be inserted at strategic locations to get the listener and editor to allow you to execute or compile a sexpr. >>>>>>>> >>>>>>>> And I’m hardly a purist CompSci person. I am someone that just wants keyboard convenience. >>>>>>>> >>>>>>>> On May 27, 2026, at 13:29, Tim Bradshaw (as tfb at cley dot com) <[email protected]> wrote: >>>>>>>>> >>>>>>>> On 27 May 2026, at 20:57, David McClain <[email protected]> wrote: >>>>>>>>>> That seems a purist Computer Scientist’s argument. >>>>>>>>> >>>>>>>> I just can't understand why, for instance, #D/xxyyzz/ is not OK, where 'xxyyzz' is some sequence containing the syntax you are interested in and '/' is any character. I mean, only a purist computer scientist would object to the delimiters, surely? >>>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Lisp Hug - the mailing list for LispWorks users >>>>>>>> [email protected] >>>>>>>> http://www.lispworks.com/support/lisp-hug.html >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Lisp Hug - the mailing list for LispWorks users >>>>>>>> [email protected] >>>>>>>> http://www.lispworks.com/support/lisp-hug.html >>>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Lisp Hug - the mailing list for LispWorks users >>>>>> [email protected] >>>>>> http://www.lispworks.com/support/lisp-hug.html >>>>>> >>>>> >>>>> _______________________________________________ >>>>> Lisp Hug - the mailing list for LispWorks users >>>>> [email protected] >>>>> http://www.lispworks.com/support/lisp-hug.html >>>> >>>> >>>> --Apple-Mail=_208CC253-8B35-44E4-8DFB-C14FAAE3A967 >>>> Content-Transfer-Encoding: quoted-printable >>>> Content-Type: text/html; >>>> charset=utf-8 >>>> >>>> <html aria-label="message body"><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">Oh wow! Very cool! Thanks for the tips, Martin. I accidentally discovered those two characters for just the purpose originally intended. What a lucky random dart!<div><br></div><div>My main puzzlement arises from the use of Terminating vs Non-Terminating macro chars. I have always built lexical analyzers based on prefix chars, not termination chars. I see that the Lisp way works. But it also implies that code that looks cramped like this would pass muster:</div><div><br></div><div><font face="Monaco">(let((c #\:))(find c"my string has colons :::":from-end t))</font></div><div><font face="Monaco"><br></font></div><div><font face="Monaco">It does, and I suppose it’s okay, but it violates common sense style.</font></div><div><font face="Monaco"><br></font></div><div! >> ! >> <! >>> font face="Monaco">BTW… I have never used Perl. Is that de rigueur among programmers of today? (I’m not a programmer). I have used SED and AWK about 40-50 years ago. But Lisp can do everything we need. Why go outside the nice environment into Bash land?<br id="lineBreakAtBeginningOfMessage"></font><div><br><blockquote type="cite"><div>On May 28, 2026, at 08:33, Martin Simmons <[email protected]> wrote:</div><br class="Apple-interchange-newline"><div><div>They didn't actually escape notice. #\U+2032 and #\U+2033 are unicode "prime"<br>characters, so software doesn't typically use them as quotes. According to<br>the authors of Wikipedia, these primes are the correct characters to use for<br>arcminutes and arcseconds, so using them is good.<br><br>We have reader macros for #\U+2018 (as a backquote), #\U+2019 (as a quote) and<br>#\U+201C (as a double-quote, paired with #\U+201D) because some software auto<br>converts the ascii backquote, quote and dou! > b! >> l! >>> e-quote into them, so it is<br>convenient to recognize them fo! >>> r cut and paste. Ironically, it looks your<br>email client did this auto conversion in your keyboard sequences below!<br><br>-- <br>Martin Simmons<br>LispWorks Ltd<br>http://www.lispworks.com/<br><br><br><br><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">On Wed, 27 May 2026 15:29:47 -0700, David McClain (as dbm at refined-audiometrics dot com) said:<br></blockquote></blockquote></blockquote></blockquote><br>Found it!!<br><br>#\U+2032 (′) and #U+2033 (″). These two seem to have escaped notice by LW. And so they can be used in symbol names and anywhere else you like, without confusing the Listener and Editor.<br><br>I have them assigned to keyboard sequences ^z-‘ and ^z-“ for easy access. My ^z- chords also map all the Greek alphabet.<br><br><br><br><blockquote type="cite">On May 27, 2026, at 14:31, David McClain <[email protected]> wrote:<br><br>And als! > o! >> ,! >>> many times, I just want to copy/paste from a printout from some other source and have Lisp interpret what is shown. Hence the desire to have embedded quote and double quote in the angle input. To use #N/xx…/ means I have to pre-edit the printout before submitting to Lisp.<br><br>But… if we had an extensible Lexical Analyzer that could be tailored to the needs….<br><br>What Common Lisp presents is cutting edge for 1980’s and earlier. But not so much for today.<br><br><blockquote type="cite">On May 27, 2026, at 14:21, David McClain (as dbm at refined-audiometrics dot com) <[email protected]> wrote:<br><br>…and also because the Editor and Listener balks when your delimited run of chars contains an isolated or odd number of |, “, or an embedded \ that isn’t being used as an escape. This speaks to the need for those bizarre comments that have to be inserted at strategic locations to get the listener and editor to allow you to execute or compile a! >> ! >>> sexpr.<br><br>And I’m hardly a purist CompSci person. I am someone th! >>> at just wants keyboard convenience.<br><br><blockquote type="cite">On May 27, 2026, at 13:29, Tim Bradshaw (as tfb at cley dot com) <[email protected]> wrote:<br><br>On 27 May 2026, at 20:57, David McClain <[email protected]> wrote:<br><blockquote type="cite">That seems a purist Computer Scientist’s argument.<br></blockquote><br>I just can't understand why, for instance, #D/xxyyzz/ is not OK, where 'xxyyzz' is some sequence containing the syntax you are interested in and '/' is any character. I mean, only a purist computer scientist would object to the delimiters, surely?<br><br>_______________________________________________<br>Lisp Hug - the mailing list for LispWorks users<br>[email protected]<br>http://www.lispworks.com/support/lisp-hug.html<br></blockquote><br><br>_______________________________________________<br>Lisp Hug - the mailing list for LispWorks users<br>[email protected]<br>http://www.lispworks.com/support/lisp! > -! >> h! >>> ug.html<br></blockquote><br></blockquote><br><br>_______________________________________________<br>Lisp Hug - the mailing list for LispWorks users<br>[email protected]<br>http://www.lispworks.com/support/lisp-hug.html<br><br></blockquote><br>_______________________________________________<br>Lisp Hug - the mailing list for LispWorks users<br>[email protected]<br>http://www.lispworks.com/support/lisp-hug.html<br></div></div></blockquote></div><br></div></body></html> >>>> --Apple-Mail=_208CC253-8B35-44E4-8DFB-C14FAAE3A967-- >>>> >> >> >> _______________________________________________ >> Lisp Hug - the mailing list for LispWorks users >> [email protected] >> http://www.lispworks.com/support/lisp-hug.html > > > _______________________________________________ > Lisp Hug - the mailing list for LispWorks users > [email protected] > http://www.lispworks.com/support/lisp-hug.html