Re: structured unicode

Brian Alliet <[email protected]> writes:
> Looks like something like this would be handy for a lots of stuff. The
> only thing I don't like is using backticks for string literals. Using
> double quotes is obviously preferable for parsing source code. I'd say
> string literals appear far more in source code than quotations appear
> in prose (well.. at least the kind of documents that are likely to be
> marked up with structured unicode). TeX users are already used to
> using backticks and single quotes to create double quotes in their
> documents anyway.

Hrm. Yeah, I guess that's one of the parts of TeX I really despise ;)

Single quotes are definately out because they are also apostrophes
(sp?), so making people escapify them would be unacceptable.  So I
guess it's a call between double quotes, which inconveniences people
writing prose, and backticks, which inconveniences people accustomed
to C/Java (ie 95% of all programmers).

Note that I'm definately not advocating a different start and end
symbol like `this'...  that's obnoxious (I think GNU Info does that?
TeX is even worse with ``this'')... I was thinking of just `this`.

I revised things a bit to try to make the parsing spec simpler.  The
main change is:

  \command               is a command with no arguments
  \command{...}          is a command with ... as an argument
  \command[..][..]{...}  is a command with three arguments ({..} is always the last one)
  \command: x y z        is a command with x y and z as arguments (it gobbles everything up until \n)
  {x y z
\command:
blah
blurp }                  is a command with blah and blurp as arguments (if a
                         colon-command appears at the end of the line
                         it eats everything until the end of the
                         enclosing scope).  This is handy for sticking
                         stuff at the top of the document that should
                         encase the entire document; you don't need to
                         match up close-braces at the end of the
                         document.

The general idea with \command[..]{..} is that the stuff in [] should
play the role of XML attributes, while the stuff in the {..} is like
the body of a tag.  I'm going to define an official XML-to-SU mapping
at some point.  I'm also trying to figure out how to make attributes
easy to type... requiring quotes (like XML) is S*T*U*P*I*D...

  \command[attr1=val1 attr2=val2]

  \command[attr1=val1][attr2=val2]

  \command[attr1=val1,attr2=val2]

Hrm, not sure about that.  The middle one has the advantage that you
never need to escapify stuff (everything up until the = is the
attribute name and everything after that (until the close-bracket) is
the value.

I'm also contemplating using \: instead of //... it really simplifies
the description of the parsing process because then comments share the
same parsing rules as commands and then you don't need some wonky
syntax for escapifying '//'... but OTOH it's more of a divergence from
the C/C++/Java/JS syntax.

The other thing I thought of is that I think that "comments" and
"deactivated code" are really two totally different things that
shouldn't share a syntax... I think using /* .. */ to disable code is
just an ugly hack we haven't gotten rid of yet.

The big advantage is that you can use here-doc form for deactivating
code and then you can require that comments be "well formed", like
literate programming comments and javadoc comments.  This lets the
comments have internal structure.  I'd also like to have a here-doc
form of strings... perhaps something like:

  \ignore\!endofcomment  all this stuff
    is a } totally { unparsed ` block to be ignored endofcomment

  \string\!endofcomment   all this stuff
    is a } totally { unparsed ` string endofcomment

Where \!foo is a special directive that consumes everything up until
'foo' without trying to parse that stuff.

I'm not psyched about the choice of ! or \string or \ignore, but I
would like to share the here-doc operator between strings and
comments.  And I'd like to keep using the backslash to avoid
introducing new special characters (which requires new forms of
escapification).

Hrm, perhaps a command that ends with ! should consume everything
until the name of the command appears again?  That would be somewhat
symmetric with the command-ending-with-a-colon...

  \endofcomment! this is a comment endofcomment

Or maybe put the bang in twice to make the structure more visible

  \endofcomment! this is a comment !endofcomment

  \!endofcomment this is a comment endofcomment!

Hrm, what about angle brackets?

  \<foo blah bloop foo>

  \<<foo blah bloop foo>>

That actually looks kinda nice.  Visually obvious without syntax
hilighting.  Can't use \[ \{ or \( since we need those for "unparsed"
[ { and (.

The supercool part is that with a "generic parsing structure" like
this, I'm going to write an incremental editor that updates the DOM
tree as you type.  Then any language based on this syntax can do
as-you-type stuff really easily like syntax highlighting, diagramming,
and even compilation (like Eclipse, which can only do it for Java).

  - a