Re: sed bug: ASCII NUL doesn't work on the rhs of y// commands
Jim Meyering <[email protected]>
| Newsgroups | gmane.comp.gnu.utils.bugs |
|---|---|
| Message-ID | <CA+8g5KH2KN1ARgrEdhkVLLFDLQsDM5twMNhnQu+EF9w9i6FLtw@mail.gmail.com> |
On Wed, Sep 3, 2014 at 4:19 AM, Paolo Bonzini <[email protected]> wrote: > Il 03/09/2014 03:09, [email protected] ha scritto: >> Hi, >> >> The subject pretty much says it all for this bug. Compare the output of >> "echo abc | sed -e 's/b/\x00/' | hexdump -c" and "echo abc | sed -e >> 'y/b/\x00/' | hexdump -c". The s command behaves correctly (as I would >> expect, it replaces the 'b' with a NUL character), while the y command >> fails to output anything when it should print NUL, resulting in an >> output file shorter than the input was. > > Looks like the bug was introduced when "y" was extended to support > multibyte characters. The minimal patch should be to change > > int trans_len = strlen(trans[2*i+1]); > > to > > char *trans = trans[2*i+1]; > int trans_len = *trans == '\0' ? 1 : strlen(trans); > > in sed/execute.c Hi Paolo, Thanks for the suggestion. Here's a complete patch (can't reuse the name "trans" that way, and I prefer to s/int/size_t/). I expect to find the precise commit that introduced the bug, adjust the log and NEWS, and then push tomorrow.
0001-fix-y-to-work-with-NUL-in-the-RHS.patch
(application/octet-stream, 5 KB)
From c86c200faf249618c526d15e8e0a50b74d4bc0d2 Mon Sep 17 00:00:00 2001 From: Jim Meyering <[email protected]> Date: Thu, 4 Sep 2014 20:52:12 -0700 Subject: [PATCH] fix "y" to work with NUL in the RHS * sed/execute.c (do_list) [case 'y']: Handle NUL bytes in the RHS of a y/LHS/RHS/ transliteration. * testsuite/y-zero.good: New test-related files. * testsuite/y-zero.inp: * testsuite/y-zero.sed: * testsuite/Makefile.am (SEDTESTS): Add y-zero here. * testsuite/Makefile.tests: And here. * NEWS (Bug fixes): Describe it. The bug was reported by [email protected], with the execute.c change mostly by Paolo Bonzini. --- ChangeLog | 14 ++++++++++++++ NEWS | 12 +++++++++++- sed/execute.c | 3 ++- testsuite/Makefile.am | 3 ++- testsuite/Makefile.tests | 2 +- testsuite/y-zero.good | Bin 0 -> 4 bytes testsuite/y-zero.inp | 1 + testsuite/y-zero.sed | 1 + 8 files changed, 32 insertions(+), 4 deletions(-) create mode 100644 testsuite/y-zero.good create mode 100644 testsuite/y-zero.inp create mode 100644 testsuite/y-zero.sed diff --git a/ChangeLog b/ChangeLog index d54232a..c90b83c 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,17 @@ +2014-09-04 Jim Meyering <[email protected]> + + fix "y" to work with NUL in the RHS + * sed/execute.c (do_list) [case 'y']: Handle NUL bytes + in the RHS of a y/LHS/RHS/ transliteration. + * testsuite/y-zero.good: New test-related files. + * testsuite/y-zero.inp: + * testsuite/y-zero.sed: + * testsuite/Makefile.am (SEDTESTS): Add y-zero here. + * testsuite/Makefile.tests: And here. + * NEWS (Bug fixes): Describe it. + The bug was reported by [email protected], + with the execute.c change mostly by Paolo Bonzini. + 2014-07-24 Jose E. Marchesi <[email protected]> * gnulib: Update. diff --git a/NEWS b/NEWS index e3c94c7..c82830d 100644 --- a/NEWS +++ b/NEWS @@ -1,4 +1,14 @@ -Sed 4.2.2 +GNU sed NEWS -*- outline -*- + +* Noteworthy changes in release ?.? (????-??-??) [?] + +** Bug fixes + + The 'y' (transliterate) operator once again works with a NUL byte + on the RHS. + + +* Noteworthy changes in release 4.2.2 (2012-12-22) [stable] * don't misbehave (truncate input) for lines of length 2^31 and longer diff --git a/sed/execute.c b/sed/execute.c index cb84192..2e13ee2 100644 --- a/sed/execute.c +++ b/sed/execute.c @@ -1567,7 +1567,8 @@ execute_program(vec, input) if (strncmp(line.active + idx, trans[2*i], mbclen) == 0) { bool move_remain_buffer = false; - int trans_len = strlen(trans[2*i+1]); + const char *tr = trans[2*i+1]; + size_t trans_len = *tr == '\0' ? 1 : strlen (tr); if (mbclen < trans_len) { diff --git a/testsuite/Makefile.am b/testsuite/Makefile.am index c670ee0..0501617 100644 --- a/testsuite/Makefile.am +++ b/testsuite/Makefile.am @@ -22,7 +22,7 @@ SEDTESTS += \ modulo numsub numsub2 numsub3 numsub4 numsub5 0range bkslashes \ head madding mac-mf empty xbxcx xbxcx3 recall recall2 xemacs \ fasts uniq manis khadafy linecnt eval distrib 8to7 y-bracket \ - y-newline allsub cv-vars classes middle bsd stdin flipcase \ + y-newline y-zero allsub cv-vars classes middle bsd stdin flipcase \ insens subwrite writeout readin insert utf8-1 utf8-2 utf8-3 utf8-4 \ badenc inplace-hold brackets amp-escape \ help version file quiet \ @@ -95,6 +95,7 @@ EXTRA_DIST = \ xbxcx3.good xbxcx3.inp xbxcx3.sed \ xemacs.good xemacs.inp xemacs.sed \ y-bracket.good y-bracket.sed y-bracket.inp \ + y-NUL.good y-NUL.sed y-NUL.inp \ y-newline.good y-newline.sed y-newline.inp # automake makes `check' depend on $(TESTS). Declare diff --git a/testsuite/Makefile.tests b/testsuite/Makefile.tests index 7aaf622..68fd077 100644 --- a/testsuite/Makefile.tests +++ b/testsuite/Makefile.tests @@ -21,7 +21,7 @@ SKIP = :>[email protected]; exit 77 enable sep inclib 8bit 8to7 newjis xabcx dollar noeol bkslashes \ numsub head madding mac-mf empty xbxcx xbxcx3 recall recall2 xemacs \ appquit fasts uniq manis linecnt khadafy allsub flipcase space modulo \ -y-bracket y-newline insert brackets amp-escape:: +y-bracket y-newline y-zero insert brackets amp-escape:: $(SEDENV) $(SED) -f $(srcdir)/[email protected] \ < $(srcdir)/[email protected] | $(TR) -d \\r > [email protected] $(CMP) $(srcdir)/[email protected] [email protected] diff --git a/testsuite/y-zero.good b/testsuite/y-zero.good new file mode 100644 index 0000000000000000000000000000000000000000..659b72404b70ab54da8f878f31930baac622ca49 GIT binary patch literal 4 LcmYdfNag|n0$2dg literal 0 HcmV?d00001 diff --git a/testsuite/y-zero.inp b/testsuite/y-zero.inp new file mode 100644 index 0000000..8baef1b --- /dev/null +++ b/testsuite/y-zero.inp @@ -0,0 +1 @@ +abc diff --git a/testsuite/y-zero.sed b/testsuite/y-zero.sed new file mode 100644 index 0000000..6b0af7b --- /dev/null +++ b/testsuite/y-zero.sed @@ -0,0 +1 @@ +y/b/\x00/ -- 2.0.0.421.g786a89d