Parsing across multiple buffers and maintaining state (prefably without additional memory usage)

Suan Yeo <[email protected]> Tue, 30 Jun 2009 22:41:22 +0000 (UTC)
Newsgroups gmane.comp.lex.flex.general
Message-ID <[email protected]>
I'm trying to search for a particular string within a chain of buffers, say,
"sausage". Right now, it works fine if "sausage" is wholly in one buffer, but if
it is split between 2 buffers, eg, {"........sau", "sage........", ...}, flex
does not match it. This is my current implementation:

while (buf)
{
  buf_state = yy_scan_bytes(...);
  result = yylex();
  yy_delete_buffer(buf_state);
  buf = buf->next;
}


I use yy_scan_bytes because as I understand it does not copy it into a new
buffer (ie. scans in-place). How should I change my implementation to achieve
the above (other than copying everything into a single buffer)?