Re: a lexilla lexer crashes

Hi,

> On Aug 3, 2025, at 3:02 PM, ISTICG BERRECHID <[email protected]> wrote:
> 
> Hi, thank you for answering,
> I choose to go with the original lua lexer and indeed got it to work with SciTE!
> I was also able to identify the source of the problem: recursive embedding
> 
> See, typst allows you to embed 'code' blocks within markup, and within that code block, you can also embed markup and so on
> For example:
> `
> This is a *markup* text. #{
> // This is code
> let x = [
>   This is text
>   #let z = "This is code"
> ]
> };
> `
> Keywords inside code blocks don't need the pre '#', but do when in markup
> 
> I believe Scintillua doesn't allow this? If not is there any intention to implement it?

I have not found a way to get recursive embedding to work. It’s a hard problem to solve.

Cheers,
Mitchell

> On Sunday, August 3, 2025 at 5:46:59 PM UTC+1 Mitchell wrote:
> Hi, 
> 
> > On Aug 3, 2025, at 12:18 PM, ISTICG BERRECHID <[email protected]> wrote: 
> > 
> > hello, ive been trying to get a lexer for 'typst' to work on scintilla but no luck, 
> > basically i found the lexer in here: 
> > https://github.com/orbitalquark/scintillua/pull/149 
> > 
> > it wasnt hard to port it to lexilla, but if i try it, it just crashes scintilla, however, disabling the embedding lines fixes it, but now the markup grammar doesnt get embedded in code grammar and vice versa 
> > 
> > is the problem relative to the library or the implementation of the lexer itself??? 
> 
> It’s a problem with your lexer implementation. 
> 
> Lexilla and Scintillua (where you found the lexer from) approach lexing in very different ways. I’m surprised it “wasn’t hard” to port a Scintillua Lua lexer to a Lexilla C++ lexer. Lexilla does not support embedded lexers like Scintillua does. You’d have to lex Markdown from within your Typst lexer, much like the HTML lexer lexes embedded languages like PHP. 
> 
> Since you’ve written the lexer and you’re getting a crash, I’d use a debugger to inspect why the lexer is crashing. The Scintilla documentation (https://scintilla.org/ScintillaDoc.html) has some links on how to write lexers that could be helpful. I would start with a basic lexer that works/doesn’t crash and then slowly add Typst features until you have the desired functionality. 
> 
> > (ps: i tried using the same lexer as is using some automated tests with something i found called scintillua, but it just halts!) 
> 
> Scintillua can drop into an existing Scintilla-based editor (like SciTE) with some configuration, so you should be able to get it to work with the Typst lexer you found. Feel free to open up a new discussion if you’d like help in that regard (https://github.com/orbitalquark/scintillua/discussions). 
> 
> Cheers, 
> Mitchell
> 
> -- 
> You received this message because you are subscribed to the Google Groups "scintilla-interest" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
> To view this discussion visit https://groups.google.com/d/msgid/scintilla-interest/1d397e8b-1dea-4332-80bd-fb2325021dd7n%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups "scintilla-interest" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion visit https://groups.google.com/d/msgid/scintilla-interest/740A84B6-5515-4074-9C63-604CFB0FA5A3%40foicica.com.