Re: Trying to understand the "rx" macro.

Hi!

Igor Hjelmstrom Vinhas Ribeiro <[email protected]> writes:

>   One suggestion: please include a chunker to go over Scheme ports in
> the API - I
> think this is useful enough to warrant it...

This has been suggested before, but there are some issues
with implementing it.

>  Here is a draft for the above idea that works:
>
> ;; Chunker to go over ports. Call this function to get an instance.
> (define (port-chunker)
>   (let ((get-next (lambda (p)
>                     (match-let (((port char-num next-chunk) p))
>                                (or next-chunk
>                                    (if (not (eof-object? (peek-char port)))
>                                        (let ((result (list port
> (read-char port) #f)))
>                                          (set-car! (list-tail p 2) result)
>                                          result)
>                                        #f)))))
>         (get-string (lambda (p)
>                       (match-let (((port ch next-chunk) p))
>                                  (if ch (string ch) "")))))
>     (make-irregex-chunker get-next get-string)))

The problem here is that when the object isn't found in the
port, you use up the whole port.  That may still be usable,
but both the DFA and backtracking backends will read ahead
in the stream to try and find match the longest possible
sequence, so it may end up reading ahead but not using some
arbitrary text after the final match.

If you were to limit this to DFAs and change the semantics
to match the _shortest_ possible sequence, you could extract
exactly one match from the port without reading any extra
characters.

>   A question about this (this is in the documentation for the
> user-provided get-next function):
>       "It must return an eq? identical object when called multiple
> times on the same chunk,"
>   Is there any way this restriction could be lifted? Alternatively,
> could we allow the user (maybe
> optionally) to pass on a predicate to be used instead of eq? to compare chunks?

Possibly, let me think about this.

>   BTW, there is a small issue in the irregex-fold/chunked function and
> another glitch in
> make-irregex-chunker one. I included two patches to fix them - see
> below for details.

Thanks!  The first bug has already been fixed in the latest
version:

  http://synthcode.com/scheme/irregex/irregex-0.7.4.tar.gz

The second patch I've applied.  I'll try to put together a
new release soon.

-- 
Alex