[9/18] argv_ref speedup: use single-argument back-references as input

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Next in the series.  This patch finally starts using back-references
(stage 8 set up the hooks, but never used them; in fact, stage 8 on head
had a stale variable reference that could cause core dumps if
back-references had been used).  Here, the references only live until the
input engine rescans the text, so there is still no overall reuse of
scanned text in future macro expansion.  But since there is less copying
of data, there is a slight speedup in operation.  On the other hand, I
also added the notion of inlining short text - there is no point in
wasting 24 bytes or more in describing a reference to a short piece of
text, so I added a threshold at which references are created.  This patch
uses more memory, particularly when boxing and unboxing arguments, because
of keeping previous arguments alive, and the extra memory usage
counteracts the speedup of less copying.  Future patches in the series
reduce how much memory is kept alive, by compacting a $@ reference into a
single entity instead of the current approach of one reference per
argument in $@.

I did encounter one difference between the branch and head; on the branch,
$0 is shared with the symbol in the symbol table alongside its expansion
value, so its lifetime is valid throughout expand_macro but not longer.
On head, the symbol table stores symbol names independently of the stack
of expansion values, the symbol table string could be invalidated even
before expand_macro completes, thus we already had been copying $0 onto
the argument stack, which means that $0 can live as long as the rest of
the arguments.  This difference is highlighted between the two
macro.c:push_arg function implementations.

This patch also adds some test of embedded NUL behavior.  I noticed that
ever since stage 3, when the input engine tracks length instead of
NUL-terminated text, that behavior of builtins on embedded NUL is changing
(in my opinion, for the better).  Adding tests will document each change,
and ensure no regressions.

On a side note, I'm headed on vacation for two weeks.  I will have limited
email access, but will not be making any commits during that time.

2007-12-21  Eric Blake  <[email protected]>

	Stage 9: share rather than copy single-arg refs.
	* m4/gnulib-cache.m4: Import quote and memmem modules.
	* src/m4.h (arg_scratch): New prototype.
	* src/input.c (INPUT_INLINE_THRESHOLD): New define.
	(push_token): Use it to inline short text, and save references to
	longer text.
	(input_init, next_token): Simplify obstack handling.
	* src/macro.c (expand_argument): Likewise.
	(expand_macro): Track scratch space.
	(arg_scratch): New function.
	(make_argv_ref): Use it.
	(push_args): Likewise, and simplify comma handling, since most
	separators are short enough to be inlined.
	* src/builtin.c (builtin_init): Avoid cast.
	(m4_errprint, m4_index): Transparently support NUL.
	(m4_translit): Use scratch space, rather than leaking memory on
	expansion stack.
	* doc/m4.texinfo (Syntax): Add new test of embedded NUL.
	* checks/get-them: Extract test that uses external files.
	* checks/check-them: Run the new test.  Use unified diff if
	possible, and force text mode when debugging NUL handling.
	* examples/null.m4: New file.
	* examples/null.out: Likewise.
	* examples/null.err: Likewise.
	* examples/Makefile.am (EXTRA_FILES): Distribute these files.
	* .gitattributes: Treat new files as text, in spite of embedded
	NUL.

- --
Don't work too hard, make some time for fun as well!

Eric Blake             [email protected]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHa1Eg84KuGfSFAYARAlegAJ9T6poBv8wIOw3iAfxWONQfYTbbogCgu237
G4bfQ8AVvAtrUQT1p12GsRI=
=ByFN
-----END PGP SIGNATURE-----

_______________________________________________
M4-patches mailing list
[email protected]
http://lists.gnu.org/mailman/listinfo/m4-patches