Re: Trying to understand the "rx" macro.

Alex,

   Irregex is *really* nice. So good, in fact, that I dropped my pet projec=
t
of migrating an Scsh portion to SISC and I am now using it instead. I
regrettably took much more than one month to look at it, though...

  Thanks for it!

  One suggestion: please include a chunker to go over Scheme ports in
the API - I
think this is useful enough to warrant it...

 Here is a draft for the above idea that works:

;; Chunker to go over ports. Call this function to get an instance.
(define (port-chunker)
  (let ((get-next (lambda (p)
                    (match-let (((port char-num next-chunk) p))
                               (or next-chunk
                                   (if (not (eof-object? (peek-char port)))
                                       (let ((result (list port
(read-char port) #f)))
                                         (set-car! (list-tail p 2) result)
                                         result)
                                       #f)))))
        (get-string (lambda (p)
                      (match-let (((port ch next-chunk) p))
                                 (if ch (string ch) "")))))
    (make-irregex-chunker get-next get-string)))

;; Sample usage of the above chunker.
(irregex-fold/chunked sre kons '() (port-chunker) (list port #f #f))

  A question about this (this is in the documentation for the
user-provided get-next function):
      "It must return an eq? identical object when called multiple
times on the same chunk,"
  Is there any way this restriction could be lifted? Alternatively,
could we allow the user (maybe
optionally) to pass on a predicate to be used instead of eq? to compare chu=
nks?

  BTW, there is a small issue in the irregex-fold/chunked function and
another glitch in
make-irregex-chunker one. I included two patches to fix them - see
below for details.

Best regards,
 Igor.

commit 786facf4dd6e771b0b5231f5cec853d58e7e40d8
Author: Igor Hjelmstrom Vinhas Ribeiro <[email protected]>
Date:   Wed Aug 12 21:17:30 2009 -0400

    Fixed call to finish procedure in irregex-fold/chunked - it
    was being passed 3 parameters, while 2 is what the default
    implementation inside this function accepts and what is
    specified in irregex documentation.

diff --git a/iasylum/irregex/irregex-0.7.3/irregex.scm
b/iasylum/irregex/irregex-0.7.3/irregex.scm
index 9d32bd9..c24be66 100644
--- a/iasylum/irregex/irregex-0.7.3/irregex.scm
+++ b/iasylum/irregex/irregex-0.7.3/irregex.scm
@@ -3177,10 +3177,10 @@
     (irregex-match-chunker-set! matches cnk)
     (let lp ((start start) (i i) (acc knil))
       (if (not start)
-          (finish start i acc)
+          (finish i acc)
           (let ((m (irregex-search/matches irx cnk start i matches)))
             (if (not m)
-                (finish start i acc)
+                (finish i acc)
                 (let* ((acc (kons start i m acc))
                        (end-src (irregex-match-end-source m 0))
                        (end-index (irregex-match-end-index m 0)))

commit 75e680cc9054d23e2c6f8b9ff675447dcbec7c3b
Author: Igor Hjelmstrom Vinhas Ribeiro <[email protected]>
Date:   Sat Aug 15 00:17:10 2009 -0400

    Fixed the make-irregex-chunker implementation to use the user-provided
    get-next in the default get-substr implementation. cdr was being
    used, which worked for the sample ropes chunker provided with
    irregex, but not for other chunkers.

diff --git a/iasylum/irregex/irregex-0.7.3/irregex.scm
b/iasylum/irregex/irregex-0.7.3/irregex.scm
index c24be66..dce7016 100644
--- a/iasylum/irregex/irregex-0.7.3/irregex.scm
+++ b/iasylum/irregex/irregex-0.7.3/irregex.scm
@@ -166,7 +166,7 @@
               (lambda (cnk1 start cnk2 end)
                 (if (eq? cnk1 cnk2)
                     (substring (get-str cnk1) start end)
-                    (let loop ((cnk (cdr cnk1))
+                    (let loop ((cnk (get-next cnk1))
                                (res (list (substring (get-str cnk1)
                                                      start
                                                      (get-end cnk1)))))
@@ -176,7 +176,7 @@
                                             (get-start cnk)
                                             end)
                                  res))
-                          (loop (cdr cnk)
+                          (loop (get-next cnk)

On Sun, Nov 9, 2008 at 16:22, Igor Hjelmstrom Vinhas
Ribeiro<[email protected]> wrote:
> Alex,
>
> =A0=A0 While I heard of irregex before, I wasn't aware that it is a super=
set of
> SRE's.
>
> =A0=A0 Thanks for bringing this to my attention - I will look at both the=
 design
> and the
> code - you will hear again from me in a month or so.
>
> - igorhvr
>
> On Sun, Nov 9, 2008 at 8:58 AM, Alex Shinn <[email protected]> wrote:
>>
>> Note that `rx' in SCSH is just a translator from SREs to
>> POSIX strings.
>>
>> There's now a fully portable, native Scheme implementation
>> of SREs available at:
>>
>> =A0http://synthcode.com/scheme/irregex/
>>
>> It has many, many more features, and includes DFA
>> conversion.
>>
>> Even if you don't want to use this code itself, I'd
>> appreciate any feedback on the extensions I've made to the
>> SRE syntax. =A0There should be a single SRE standard (to which
>> effect I'm debating writing a SRFI).
>>
>> --
>> Alex
>>
>>
>> "Igor Hjelmstrom Vinhas Ribeiro" <[email protected]>
>> writes:
>>
>> > Brian and Rolf,
>> >
>> > =A0 =A0Thanks for the excellent explanation and for the pointer
>> > to William's ps.gz file. I now understand how the macro works
>> > and how I will go forward.
>> >
>> > Best Regards,
>> > =A0Igor.
>> >
>> > On Sun, Nov 2, 2008 at 6:34 PM, RT Happe <[email protected]> wrote:
>> >>
>> >> Brian D. Carlstrom wrote [a short explanation of s48 low-level macros=
].
>> >>
>> >> Addendum: I don't seem to find the paper on explicit renaming macros,
>> >> to be read as an introduction to the s48 low level macro facility,
>> >> in the scsh/s48 tarball, but the .ps.gz
>> >>
>> >> =A0William D Clinger. Hygienic macros through explicit renaming
>> >> =A0http://www.ccs.neu.edu/home/will/papers.html
>> >>
>> >> should do the job as well.
>> >>
>> >>
>> >>> symbols. the third comparator argument function is less frequently
>> >>> used
>> >>> as the gensym function, but I believe it is used to compare symbols
>> >>> for
>> >>> equality taking into account any newly gensymed symbols, etc.
>> >>
>> >> To avoid terminological confusion: the "symbols/gensyms" aren't symbo=
ls
>> >> but generated names.
>> >>
>> >>
>> >> rt
>> >>
>> >
>
>