Re: Nitpicky definition compatibility bug, SML/NJ deviation

On Fri, Aug 26, 2011 at 11:56 AM, Robert J. Simmons <[email protected]> wrote:
> I am indeed interested in that pygment highlighter!
>
> I've currently got a pull request out to Pygments for the highlighter
> I worked out (https://bitbucket.org/birkenfeld/pygments-main/pull-request/14/add-support-for-standard-ml).
> The one I wrote is decidedly more hackish in service of doing a bit
> more than just lexing  - for instance,
> http://typesafety.net/tempbovik/example.html and
> http://typesafety.net/tempbovik/intsyn.fun.html (note: not permanent
> links) are examples of the special treatment of datatypes and function
> definitions. I'll look into seeing if I can combine the advantages of
> both.

In my Enscript lexer (http://mlton.org/Enscript), I tried to do some
more semantic highlighting.  I'm not sure that it is worth the effort.
 Unless you actually parse (and, for many languages, also deal
properly with imports and elaborate/type-check), you are always going
to fall short of the actual language.  And, so long as these syntax
highlighters are so heavily biased towards regexps, its hard to do
actual parsing.  (I ranted a little about the differences between a
"traditional" lexer generator and RE-based syntax highlighters in the
previously linked mail message).

For instance, why highlight identifiers bound by "fun", but not other
identifiers bound by "val"?

> Another question: what's MLton doing here? SML of New Jersey rejects
> this, and I tend to agree with it.
>
> fun testfun1 ('a):''a = x
> fun testfun1 (x:''a):'a = x
>
> I came across this originally when I tried to use a tyvarseq (', '',
> ''', '''') in a datatype declaration and MLton complained that my type
> variables were't pairwise distinct.

That does look like a bug.  HaMLet (which is usually my goto for
"who's right?" questions), sort of sides with SML/NJ (it rejects),
although whereas SML/NJ complains about the same type variable used
with different equality status, HaMLet complains about not matching
the constraint:

SML/NJ:
- fun f (x: ''a) : 'a = x;
stdIn:1.5-1.24 Error: type variable a occurs with different equality
properties in the same scope

HaMLet:
- fun f (x: ''a) : 'a = x;
(input 1):1.4-1.23: expression does not match annotation

HaMLet seems to treat 'a, ''a, '''a, ... as distinct type variables
(all but the first having equality status).  That seems to be the
natural thing to do.
SML/NJ and MLton seem to treat 'a and ''a as "related" type variables;
SML/NJ complains while MLton seems to collapse them down to
non-equality:

[mtf@graywolf tmp]$ cat - > z.sml
fun f (x: ''a) : 'a = x
val z = f (fn () => ())
[mtf@graywolf tmp]$ mlton z.sml

[mtf@graywolf tmp]$ cat - > z.sml
fun f (x: ''a) : ''a = x
val z = f (fn () => ())
[mtf@graywolf tmp]$ mlton z.sml
Error: z.sml 2.9.
  Function applied to incorrect argument.
    expects: [<equality>]
    but got: [<non-equality>]
    in: f (fn (()) => (()))