argv_ref patch 19: support builtin tokens in more macros

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

This is stage 19 of 18 :)  Actually, at this point, I am now (slowly)
merging the remaining patch series from the argv_ref speedup branch onto
the branch-1_4 and master branches.  There are still a couple of speedups
to go on the argv_ref branch before I release a stable version of m4 1.4.11.

This patch expands the framework created by earlier patches to support a
builtin token in the middle of a FIFO chain of expansion text.  Right now,
the only way to hit this scenario is through parameter expansion, and even
that is limited (defn with multiple arguments still does not support
builtin tokens alongside anything else, and builtin tokens encountered
inside comments or quotes are flattened).  There should be no memory
impact (the token_chain union did not increase in size), and maybe a
slight speed penalty due to more bookkeeping.  This patch brings GNU M4
behavior in line with both BSD and Solaris behavior on a couple of fronts:

First, all user macros now transparently handle builtin tokens in
parameter expansions, rather than flattening it to the empty string.
Therefore,
~ define(`foo', `define(`$1',$2)')foo(`bar', defn(`divnum'))bar
now results in 0 (bar defined to <divnum>) rather than '' (<divnum> was
flattened while expanding foo, and bar defined to the empty string).

Second, comparison of macro definitions is now transparent to how those
macros are implemented.  A user no longer needs to know whether a macro is
builtin or not.  For example,
~ ifelse(defn(`divnum'),defn(`dnl'),yes,no)
now results in 'no' (those two macros are quite different) rather than yes
(those two macros are builtins, both got flattened to '', and two empty
strings are equal).

Finally, this patch makes GNU M4 consistent on handling builtin tokens
encountered where a macro name was expected.  Here, there is no comparison
to other implementations (both Solaris and BSD forbid defining the empty
string as a macro name), but it is a lot nicer that if 'define' rejects a
macro name, other things like 'popdef' or 'traceon' should do so as well.

On the master branch, I further divided the patch into three parts.  The
first part caches builtins loaded from a module (also useful for a later
patch that will make tracing a builtin possible), the second swaps the
sense of the bit stating whether a builtin plans on supporting builtin
tokens as arguments (making the logic consistent that a macro must request
flattening, now that more macros handle builtins), and the third actually
fixing the input engine to support references to builtin tokens.

2008-03-14  Eric Blake  <[email protected]>

	Stage 19: allow builtin tokens in more macros.
	Allow builtin tokens inside symbol chains, although for now, they
	are not allowed inside comments or quotes.  Enable builtin token
	handling in more macros, if only to consistently diagnose invalid
	macro names.
	Memory impact: none.
	Speed impact: slight impact, due to more bookkeeping.
	* src/m4.h (enum token_chain_type): Add CHAIN_FUNC.
	(struct token_chain): Add func member.
	* src/input.c (push_token, pop_input, input_print, peek_input)
	(next_char_1, init_macro_token): Handle builtin tokens from
	back-references.
	(next_token): Flatten builtin tokens inside comments or quotes,
	except when a builtin is the only thing inside quotes.
	* src/macro.c (expand_argument): Strengthen assertion.
	(collect_arguments): Handle builtin tokens.
	(expand_macro): Fix harmless typo.
	(arg_token): Add parameter.
	(arg_type, arg_text, arg_equal, arg_empty, arg_len, arg_func)
	(arg_print, push_arg_quote, push_args): Update callers to either
	require flattened arguments or to handle builtins.
	* src/builtin.c (m4_defn, m4_dumpdef, m4_ifdef, m4_ifelse)
	(m4_popdef, m4_shift, m4_traceoff, m4_traceon, m4_undefine):
	Handle builtin tokens, either by recognizing invalid macro names
	or passing them through transparently.
	(define_user_macro): Make all user macros handle builtin token
	arguments transparently.
	* doc/m4.texinfo (Defn, Ifdef, Ifelse, Debuglen): Document and
	test the new behavior.
	* NEWS: Document this change.

- --
Don't work too hard, make some time for fun as well!

Eric Blake             [email protected]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkfaeMMACgkQ84KuGfSFAYBvZgCdGNYSMn+uRyRfBIV/XoRfvE6l
3oUAn3DKnKN3VyIOcVuIm/Wvrsz95Ywo
=srT9
-----END PGP SIGNATURE-----

_______________________________________________
M4-patches mailing list
[email protected]
http://lists.gnu.org/mailman/listinfo/m4-patches