[bug #68446] lexer double counting chars

"Dmitri A. Sergatskov" <[email protected]> Sat, 13 Jun 2026 00:38:18 -0400 (EDT)
Newsgroups	gmane.comp.gnu.octave.bugs
Message-ID	<[email protected]>
Please use the bug tracker to post updates to a bug report.  The mailing list is intended as a read-only notification stream.  Info posted to this mailing list address won't appear in the tracker database where it is most useful.

URL:
  <https://savannah.gnu.org/bugs/?68446>

                 Summary: lexer double counting chars
                   Group: GNU Octave
               Submitter: dasergatskov
               Submitted: Sat 13 Jun 2026 04:38:14 AM UTC
                Category: Octave Function
                Severity: 3 - Normal
                Priority: 5 - Normal
              Item Group: Unexpected Error or Warning
                  Status: None
             Assigned to: None
         Originator Name:
        Originator Email:
             Open/Closed: Open
         Discussion Lock: Unlocked
                 Release: dev
        Operating System: Any
           Fixed Release: None
         Planned Release: None


    _______________________________________________________

Follow-up Comments:


-------------------------------------------------------
Date: Sat 13 Jun 2026 04:38:14 AM UTC By: Dmitri A. Sergatskov <dasergatskov>

octave:1> abc)
error: syntax error

>>> abc)
       ^
octave:1> ?abc)
error: syntax error

>>> ?abc)
            ^
octave:1> foo@bar)
error: syntax error

>>> foo@bar)
                  ^

The problem appears to be double-counting: update_token_positions (yyleng)
already advances m_filepos, but the
  METAQUERY/FQ_IDENT/SUPERCLASSREF paths advanced it again with
m_filepos.increment_column (flex_yyleng ()).

(I could not figure out the FQ_IDENT example, but assume it has the same
problem from general principles :) )

So this diff seems to fixes it for me:


--- a/libinterp/parse-tree/lex.ll       Wed Jun 10 20:03:57 2026 +0200
+++ b/libinterp/parse-tree/lex.ll       Sat Jun 13 00:32:04 2026 -0400
@@ -3551,8 +3551,6 @@
 
   token *tok = new token (SUPERCLASSREF, meth, cls, m_tok_beg, m_tok_end,
get_comment_list ());
 
-  m_filepos.increment_column (flex_yyleng ());
-
   return handle_token (tok);
 }
 
@@ -3566,8 +3564,6 @@
 
   token *tok = new token (METAQUERY, cls, m_tok_beg, m_tok_end,
get_comment_list ());
 
-  m_filepos.increment_column (flex_yyleng ());
-
   return tok;
 }
 
@@ -3581,8 +3577,6 @@
 
   token *tok = new token (FQ_IDENT, ident, m_tok_beg, m_tok_end,
get_comment_list ());
 
-  m_filepos.increment_column (flex_yyleng ());
-
   return tok;
 }
 




Dmitri.
-- 








    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?68446>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/
signature.asc (application/pgp-signature, 228 B)