Re: rfc: curvbol - a lisp language for processing text (long)

Arthur Lemmens <[email protected]> writes:

> Your specification looks interesting at first sight, but would be a lot
> more readable if you included examples of each operator. Some kind of
> motivation and a comparison with other systems (why/when is this better/
> worse than regular expressions, recursive decent parsers, parser generators)
> would also help a lot.

lisp already has regexps, recursive desecent parsers and parser
generators, i'm not trying to replace/improve any of those, i'm
attempting a different way looking at the problem. instead of
describing the grammar of the file (which i many not know, or may not
be expressable as a context free grammar) or mixing lisp code with
regexp checks (which gets very messy if the logic is complicated) i
want to express how to move around in the text and say what to do with
certain sections of the text. basically this is another tool to put in
the toolbox, not a replacement for the other techniques.

Anyway here's the list of ops with examples. at the end of this
(rather long) email i've put a real life curvbol program. it is in
fact the only existing real life curvbol program :).

Before i begin, curvbol programs are called via the curvbol macro:

(curvbol subject &rest implicit-seq) - evaluate IMPLICIT-SEQ with the
subject bound to SUBJECT, cursor starts at 0, direction starts as
:forwards.

All the examples look like this:

[code]
 ==> [value], [env], [subject with cursor as #\|]

The Examples:

MATCH:

(curvbol "abcd" (match "a")) ==> "a", NIL, a|bcd

(curvbol "abcd" (match "a") (match "bc")) ==> "bc", NIL, abc|d

SEQ:

(curvbol "abcd"
  (seq
    (match "a")
    (match "bc")))
==> "bc", NIL, abc|d

(curvbol "abcd"
  (seq
    (match "a")
    (match "X")))
==> NIL, NIL, |abcd

LOOP:

(curvbol "aa.aa.aa."
  (loop (0 nil t)
    (match "aa.")))
==> "aa.", NIL, aa.aa.aa.|

(curvbol "aa.aa.aa."
  (loop (0 nil nil)
    (match "aa."))
  (match "aa."))
==> "aa.", NIL, aa.|aa.aa.

(curvbol "aa.aa.aa."
  (loop (0 nil t)
    (match "aa."))
  (match "aa."))
==> "aa.", NIL, aa.aa.aa.|

GOTO:

(curvbol "abcd"
  (goto (match "bc"))
  (match "d"))
==> NIL, NIL, |abcd

(curvbol "abcd"
  (goto (match "bc"))
  (match "bc"))
==> "bc", NIL, abc|d

GOPAST:

(curvbol "abcd"
  (gopast (match "bc")))
==> "bc", NIL, abc|d

OR:

(curvbol "abcd"
  (or 
    (match "X")
    (match "a")))
==> "a", NIL, a|bcd

(curvbol "aa"
  (or 
    (match "aa")
    (match "a"))
  (match "a"))
==> "a", NIL, aa|bb

(curvbol "abc"
  (or
    (seq
      (gopast (match "b"))
      (match "X"))
    (seq
      (gopast (match "b"))
      (match "c"))))
==> "c", NIL, abc|

AND:

(curvbol "abcd"
  (and
    (match "a")
    (match "b")))
==> NIL, NIL, |abcd

(curvbol "abcd"
  (and
    (goto (match "c"))
    (goto (match "b"))
    (gopast (match "a"))))
==> "a", NIL, a|bcd

REVERSE:

(curvbol "abcd"
  (goto (match "cd"))
  (reverse
    (match "ab")))
==> "ab", NIL, |abcd

(curvbol "abcd"
  (gopast (match "d"))
  (reverse
    (goto (match "b"))
    (match "a")))
==> "a", NIL, |abcd

BACKWARDS:

(curvbol "abcd"
  (backwards
    (match "bcd")))
==> "bcd", NIL, |abcd

(curvbol "abcd"
  (backwards
    (match "d")
    (match "c"))
  (match "a"))
==> "a", NIL, a|bcd

NOT:

(curvbol "abcd"
  (not (match "X")))
==> T, NIL, |abcd

(curvbol "abcd"
  (not (goto (match "b"))
       (match "X")))
==> T, NIL, ab|cd

(curvbol "abcd"
  (not (goto (match "c"))
       (match "d")))
==> NIL, NIL, |abcd

TRY:

(curvbol "abcd"
  (try (goto (match "c"))
       (match "X")))
==> NIL, NIL, abc|d

(curvbol "abcd"
  (match "a")
  (try
    (goto (match "c"))
    (match "d")))
==> "d", NIL, a|bcd

TEST:

(curvbol "abcd"
  (match "a")
  (test (match "b")))
==> "b", NIL, a|bcd

(curvbol "abcd"
  (match "a")
  (test (match "X")))
==> NIL, NIL, a|bcd

HOP:

(curvbol "abcd"
  (hop 2))
==> 2, NIL, ab|cd

(curvbol "abcd"
  (hop 5))
==> NIL, NIL, |abcd

LET:

(curvbol "abcd"
  (let y "ab")
  (match y))
==> "ab", NIL, ab|cd

MAKE-SPLICE:

(curvbol "abcd"
  (make-splice 1 3))
==> "bc", NIL, |abcd

(curvbol "abcd"
  (match "a")
  (make-splice
    (cursor)
    (seq
      (gopast (match "d"))
      (cursor))))
==> "bc", NIL, abcd|

A Real Life Example:

This examples use these curvbol functions/macros which aren't part of
the core:

(chew-whitespace) - moves the cursor past any whitespace chars (#\Space,
#\Tab, #\Newline, #\Linfeed, or  #\Return). Perl equivalent: '\s*'

(let-splice var &rest seq) - a macro which binds var to a splice which
starts at cursor's value before the evaluation of SEQ and ends at
cursor's value after the evaluation of SEQ.

(defcurvbol parse-scheda ()
  ;; the body of the curvbol function is an implicit SEQ, the
  ;; implication is that if any of these forms fail (return NIL) the
  ;; function will immediatly return NIL

  (gopast (match "Padiglione"))
  (gopast (match "<FONT color=\"red\">"))
  (chew-whitespace)

  (let-splice padiglione
    ;; move the cursor to the next non #\Space or digit char
    (goto (not (class " 0123456789"))))

  (gopast (match "<b><span style=\"background-color: #000000\"><font color=\"#FFFFFF\">")
          (chew-whitespace))

  ;; the cursor is now located after any whitespace after the above string
  (let-splice ragione-sociale
    ;; move the cursor to the start of the next sequence of white
    ;; space chars followed by "<BR>"
    (goto (chew-whitespace)
          (match "<BR>")))

  ;; find the address
  (gopast (match "<font >") (chew-whitespace))
  (let-splice indirizzo
    (goto (chew-whitespace) (match "<BR>")))

  ;; (or X t) is basically the '?' regepx operator 
  (or
   (seq
    ;; move the cursor to the start of next sequence of digits
    (goto (class "0123456789"))
    (let-splice cap
      (goto (not (class "0123456789"))))
   t)

  (gopast (match "-") (chew-whitespace))

  (or
   (let-splice citta
     (goto (match "-")))
   t)

  (gopast (match "-") (chew-whitespace))

  (let-splice paese
    (goto (chew-whitespace) (match "-")))

  ;; in the subject we don't know which, if any, of these will appear
  ;; nor do we know in what order, but since we're using the AND op each
  ;; clause will be evaluated starting at the same cursor position.
  (and

   (seq
    (gopast (match "Tel."))
    (let-splice tel
      (goto (not (class "0123456789-+ ()/")))))

   (seq
    (gopast (match "Fax"))
    (let-splice fax (goto (not (class "0123456789-+ ()/")))))

   (seq
    (gopast (match "<TD WIDTH =\"305\"> E-Mail: "))
    (let-splice email (goto (match "</TD>")))))

  (or
   (seq
    (gopast (match "Stand:"))
    (gopast (match "</b>"))
    (let-splice desc
      (goto (match "<a HREF=\"javascript:newpage('dettaglioAzienda.do"))))
   t))

well, at least this takes care of the docs :)

-- 
-Marco
Ring the bells that still can ring.
Forget your perfect offering.
There is a crack in everything.
That's how the light gets in.
     -Leonard Cohen