Re: a lexilla lexer crashes

Hi, thank you for answering,
I choose to go with the original lua lexer and indeed got it to work with 
SciTE!
I was also able to identify the source of the problem: recursive embedding

See, typst allows you to embed 'code' blocks within markup, and within that 
code block, you can also embed markup and so on
For example:
`
This is a *markup* text. #{
// This is code
let x = [
  This is text
  #let z = "This is code"
]
};
`
Keywords inside code blocks don't need the pre '#', but do when in markup

I believe Scintillua doesn't allow this? If not is there any intention to 
implement it?
On Sunday, August 3, 2025 at 5:46:59 PM UTC+1 Mitchell wrote:

> Hi,
>
> > On Aug 3, 2025, at 12:18 PM, ISTICG BERRECHID <[email protected]> wrote:
> > 
> > hello, ive been trying to get a lexer for 'typst' to work on scintilla 
> but no luck,
> > basically i found the lexer in here:
> > https://github.com/orbitalquark/scintillua/pull/149
> > 
> > it wasnt hard to port it to lexilla, but if i try it, it just crashes 
> scintilla, however, disabling the embedding lines fixes it, but now the 
> markup grammar doesnt get embedded in code grammar and vice versa
> > 
> > is the problem relative to the library or the implementation of the 
> lexer itself???
>
> It’s a problem with your lexer implementation.
>
> Lexilla and Scintillua (where you found the lexer from) approach lexing in 
> very different ways. I’m surprised it “wasn’t hard” to port a Scintillua 
> Lua lexer to a Lexilla C++ lexer. Lexilla does not support embedded lexers 
> like Scintillua does. You’d have to lex Markdown from within your Typst 
> lexer, much like the HTML lexer lexes embedded languages like PHP.
>
> Since you’ve written the lexer and you’re getting a crash, I’d use a 
> debugger to inspect why the lexer is crashing. The Scintilla documentation (
> https://scintilla.org/ScintillaDoc.html) has some links on how to write 
> lexers that could be helpful. I would start with a basic lexer that 
> works/doesn’t crash and then slowly add Typst features until you have the 
> desired functionality.
>
> > (ps: i tried using the same lexer as is using some automated tests with 
> something i found called scintillua, but it just halts!) 
>
> Scintillua can drop into an existing Scintilla-based editor (like SciTE) 
> with some configuration, so you should be able to get it to work with the 
> Typst lexer you found. Feel free to open up a new discussion if you’d like 
> help in that regard (
> https://github.com/orbitalquark/scintillua/discussions).
>
> Cheers,
> Mitchell

-- 
You received this message because you are subscribed to the Google Groups "scintilla-interest" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion visit https://groups.google.com/d/msgid/scintilla-interest/1d397e8b-1dea-4332-80bd-fb2325021dd7n%40googlegroups.com.