argv_ref patch 24: allow NUL in macro names

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Next in the series.  This starts a subseries of patches to allow
transparent NUL handling through more of m4.  POSIX is silent on the
matter (m4 is only required to handle text files, and the presence of an
embedded NUL automatically disqualifies a file as text), and no other m4
implementation handles NUL characters sanely, but GNU Coding Standards
recommend being transparent when possible.  This patch focuses on allowing
NUL in a macro name.

I'm still not settled on how to generate NUL from within m4 - I'm thinking
that adding \ escapes to format might be the way to go, but don't want to
break backwards compatibility if it can be helped.  So all of the tests in
argv_ref that deal with NUL rely on printf(1) or a pre-formed file with
embedded NUL.

This patch doesn't really use too much extra memory, and the extra
bookkeeping for handling NUL might cause a slight slowdown but not
drastic.  [On the side, cygwin's strlen and memchr functions are now
several times faster on unaligned data as a result of evaluating the
timing impacts - word operations are faster than byte operations on modern
hardware.]  On the master branch, I split it into three patches -
reworking m4_error to take a call_info * instead of a char * (so that the
length can be passed in alongside the name), changing the symbol table to
support NUL, and changing all macro name output (warnings, dumpdef, trace)
to handle or quote difficult characters while avoiding extra munging of
the global location variables.  I still might add a debugmode(b) option
later on that enables \ quoting of dumpdef/trace output, and possibly even
make debugfile turn 'b' on or off automatically based on whether the new
output file is a tty, but that is not in argv_ref yet.

2008-06-02  Eric Blake  <[email protected]>

	Stage 24: Allow embedded NUL in macro names.
	Replace const char *macro_name with const m4_call_info *call.  Use
	length rather than NUL-termination when tracking macro names,
	including in diagnostics.  Quote instances of problematic
	characters in macro names when presented to user.
	Memory impact: none.
	Speed impact: slight penalty, due to more bookkeeping.
	* m4/gnulib-cache.m4: Import xmemdup0 module.
	* src/m4.h (define_builtin, push_string_init, push_wrapup_init)
	(lookup_symbol): Add parameters.
	(struct call_info): Add name_len member.
	(struct token_data) [ENABLE_CHANGEWORD]: Add original_len member.
	(struct symbol): Add len member.
	(TOKEN_DATA_ORIG_LEN, SYMBOL_NAME_LEN): New accessors.
	(m4_error_at_line, m4_warn_at_line): Delete.
	(m4_error, m4_warn, debug_set_output, skip_line, set_word_regexp)
	(bad_argc, evaluate): Adjust parameter type.
	* src/m4.c (m4_error_at_line, m4_warn_at_line): Delete.
	(m4_verror_at_line, m4_error, m4_warn): Adjust parameter type.
	(main): Adjust caller.
	* src/symtab.c (profile_strcmp) [DEBUG_SYMTAB]: Rename...
	(profile_memcmp): ...and accomodate NUL.
	(hash, lookup_symbol): Add parameter to track length.
	(symtab_debug, symtab_print_list): Adjust callers.
	* src/input.c (push_string_init, push_wrapup_init): Take location
	as parameter rather than using global state.
	(skip_line, set_word_regexp, next_token): Adjust parameter type.
	(peek_input, next_char_1, match_input, lex_debug): Adjust
	callers.
	* src/macro.c (struct macro_arguments): Delete argv0 and argv0_len
	members, now covered by info.
	(expand_input, expand_token, expand_argument, collect_arguments)
	(expand_macro, arg_text, arg_empty, arg_len, make_argv_ref)
	(push_arg, wrap_args): Adjust callers.
	* src/builtin.c (define_builtin): Add parameter.
	(bad_argc, numeric_arg, mkstemp_helper, substitute): Adjust
	parameter type.
	(define_user_macro, builtin_init, define_macro, m4_undefine)
	(m4_popdef, m4_ifdef, m4_ifelse, dumpdef_cmp, m4_dumpdef)
	(m4_builtin, m4_indir, m4_defn, m4_syscmd, m4_esyscmd, m4_eval)
	(m4_incr, m4_decr, m4_divert, m4_divnum, m4_undivert, m4_shift)
	(m4_changequote, m4_changecom, m4_changeword, include)
	(m4_maketemp, m4_mkstemp, m4_errprint, m4___file__, m4___line__)
	(m4___program__, m4_m4exit, m4_m4wrap, m4_traceon, m4_traceoff)
	(m4_debugmode, m4_debugfile, m4_len, m4_index, m4_substr)
	(m4_translit, m4_format, m4_regexp, m4_patsubst): Adjust all
	callers.
	* src/debug.c (debug_set_file, debug_set_output): Adjust parameter
	type.
	(trace_flush, trace_prepre, trace_pre): Adjust all callers.
	* src/eval.c (logical_or_term, logical_and_term, or_term)
	(xor_term, and_term, equality_term, cmp_term, shift_term)
	(add_term, mult_term, exp_term, unary_term, simple_term)
	(evaluate): Adjust parameter type.
	* src/format.c (arg_int, arg_long, arg_double): Likewise.
	(expand_format): Adjust caller.
	* doc/m4.texinfo (Using frozen files): Test this.
	* examples/null.m4: Likewise.
	* examples/null.err: Adjust expected output.
	* examples/null.out: Likewise.

- --
Don't work too hard, make some time for fun as well!

Eric Blake             [email protected]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Public key at home.comcast.net/~ericblake/eblake.gpg
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkhEv6IACgkQ84KuGfSFAYCIjgCbBAPRTREJcR1B2cxsq+1d4x4u
fEMAnix0icRiKPEEXHqF0fhxlQ9g+HFP
=yDU0
-----END PGP SIGNATURE-----

_______________________________________________
M4-patches mailing list
[email protected]
http://lists.gnu.org/mailman/listinfo/m4-patches