Re: [SC22WG14.37362] alx-0097r0 - <memory.h>, the legitimate header for memcpy(3) et al.
Alejandro Colomar <[email protected]> Fri, 31 Jul 2026 22:27:36 +0200
| Newsgroups | gmane.comp.lib.gnulib.bugs |
|---|---|
| Message-ID | <amz8nT6CpAlIVIKo@devuan> |
[CC += gnulib; I attach the draft proposal under discussion, so that gnulib maintainers can read it] Hi Keith, > Date: 2026-07-31 10:32:58-0700 > From: Keith Bostic <[email protected]> > > On Fri, Jul 31, 2026 at 8:00 AM Joseph Myers <[email protected]> wrote: > > > On Fri, 31 Jul 2026, Alejandro Colomar wrote: > > > > > Here's a draft of a proposal for re-introducing <memory.h>, and for > > > introducing a new <nonstring.h>. <string.h> is specified to include > > > both of them, and so it won't break existing code. > > > > I don't think the include files should change. The existing ones will remain effectively forever, indeed. <string.h> and its content aren't going anywhere. But should/shouldn't we standardize old-but-new headers that can be used too? They are after all supported by almost-all POSIX-ish libc implementations. > I do think sending a signal to future programmers is a legitimate goal. Thanks! > First, the current header contents are frozen forever. We can't break that > much software, so changes can only add to the pile. Indeed; that's the proposal. > Second, the Linux man pages are probably the best way to accomplish this > goal. Indeed; right now, I'll document mem*() as being in <memory.h>, plus a paragraph in STANDARDS clarifying that <string.h> is the standard header, but <memory.h> came first. I'll CC you when I send the patch to the mailing list in a moment. Here's a sample of what it changes (I diff(1)d the formatted page, so everyone can understand the change): --- HEAD^:man/man3/memchr.3 +++ HEAD:man/man3/memchr.3 @@ -7,7 +7,7 @@ LIBRARY Standard C library (libc, -lc) SYNOPSIS - #include <string.h> + #include <memory.h> // See STANDARDS void *memchr(size_t n; const void s[n], int c, size_t n); @@ -34,6 +34,8 @@ ATTRIBUTES STANDARDS C11, POSIX.1‐2008. + ISO C and POSIX declare this function in <string.h>. + HISTORY POSIX.1‐2001, C89, SVr4, 4.3BSD. and here's the change for non-standard functions: --- HEAD^:man/man3/mempcpy.3 +++ HEAD:man/man3/mempcpy.3 @@ -8,7 +8,7 @@ LIBRARY SYNOPSIS #define _GNU_SOURCE /* See feature_test_macros(7) */ - #include <string.h> + #include <memory.h> void *mempcpy(size_t n; void dest[restrict n], const void src[restrict n], For the non-standard functions, they don't even get a notice; <memory.h> will be the only documented header. > Alex, you can make that change with no committee needed. Agreed. BTW, gnulib maintainers, would you mind adding the <memory.h> header for compatibility? > Changing > include files assumes file layout does work the documentation can't, and I > doubt that (a new programmer won't skim include files for function > prototypes). I've recently received several emails from people that claim that strncpy(3) is a string function because it's specified by ISO C in the subclause called "String handling <string.h>". I believe the change to the file layout would be meaningful. I agree that documentation would probably be even stronger, but if we manage to do both, it'd be the strongest signal. > Third, AI assistance drives the human cost of the current (admittedly > messy) layout to zero. The one group that might benefit from this change, > future programmers, is increasingly machines that won't care. I hope some programmers (including new ones) will continue writing programs. > And I'd be > shocked if any current AI engine failed to catch strncpy misuse on a string. Thanks for the feedback! BTW, would you mind applying/proposing a similar change to FreeBSD manual pages, if you can? Have a lovely night! Alex -- <https://www.alejandro-colomar.es>
alx-0097.txt
(text/plain, 12.4 KB)
Name alx-0097r1 - <memory.h>, the legitimate header for memcpy(3) et al. Principles - Avoid ambiguities - Codify existing practice to address evident deficiencies Category Revert regression; library Author Alejandro Colomar <[email protected]> Cc: Mark Harris <[email protected]> Cc: Nevin Liber <[email protected]> Cc: JeanHeyd Meneide <[email protected]> Cc: Joseph Myers <[email protected]> Cc: Keith Bostic <[email protected]> History <https://www.alejandro-colomar.es/src/alx/alx/wg14/alx-0097.git/> r0 (2026-07-31): - Initial draft. r1 (2026-07-31): - Avoid cd(1). Rationale The standard mixes functions for handling strings, functions for handling bytes, and other hybrids, all in a single header file: <string.h>. This has historically caused confusion, as for example leading to believe that strncpy(3) is appropriate to handle strings. Let's separate the standard clearly into three different header files that differentiate these APIs. System V It turns out that mem*() functions haven't been always in <string.h>. These functions were first introduced in 1983 in System V, and were added in a separate <memory.h> file. Very soon, they were added to 4.3BSD in 1986 for compatibility to System V, also in <memory.h>. Most modern-day POSIX-compatible libc implementations still provide this header file for backwards compatibility reasons: $ find ~/src/bsd/freebsd/main/ | grep /include/memory.h /home/alx/src/bsd/freebsd/main/include/memory.h $ find ~/src/bsd/netbsd/trunk/ | grep /include/memory.h /home/alx/src/bsd/netbsd/trunk/include/memory.h $ find ~/src/bsd/openbsd/master/ | grep /include/memory.h /home/alx/src/bsd/openbsd/master/include/memory.h $ find ~/src/gnu/glibc/master/ | grep /include/memory.h /home/alx/src/gnu/glibc/master/include/memory.h $ find ~/src/musl/libc/master/ | grep /include/memory.h /home/alx/src/musl/libc/master/include/memory.h C89 It seems that it was C89 that moved the mem*() functions to <string.h>, even though the Rationale document for C89 doesn't mention it at all. This was a huge mistake --we can see the consequences in the many misuses of these functions, and programmers that believe they are appropriate for uses that they are unappropriate-- that we can undo now. Design decisions To avoid breaking the world, let's provide the declarations of mem*() in both <string.h> and the old-but-new <memory.h>. This will allow existing programs to continue working without changes. It will also allow new programs to use the more higienic header files. This is actually what is done by existing implementations, which provide the declarations in both header files. So, this is standardization of prior art. <nonstring.h> The strn*() functions didn't have their own separate header, but to avoid miususe, let's have a third header file for them. The name comes from the GCC [[gnu::nonstring]] attribute, which is used to refer to things that are somewhat similar to a string but aren't really a string (they are not necessarily NUL-terminated). Strings are valid nonstrings, but not the other way around. This is why some strn*() functions still work well on strings, such as strncmp(3). However, that shouldn't prevent moving them to this header file. Also, some strn*() functions work with strings in some parameters, while taking nonstrings in others. For example, strncat(3) takes a nonstring, and produces a string; and strncpy(3) takes a string (or a nonstring), and produces a nonstring. Let's put all of these in <nonstring.h>, so that <string.h> will remain as a header file for functions that *exclusively* handle strings. <wchar.h> Because <wchar.h> is already more packed than <string.h>, containing also stuff that would correspond to other headers (e.g., wcstol(3)), let's not split that one. A refactor of <wchar.h> should entertain a much larger task, and create files such as <wstring.h>, <wstdlib.h>, <wmemory.h>, etc. Proposed wording Based on N3886. 7 Library @@ New subclause after 7.27 ("_Noreturn <stdnoreturn.h>") +7.<27+1> Memory handling <memory.h> +7.<27+1>.1 Memory function conventions @@ Copy 7.28.1p1 as 7.<27+1>.1p1, then: The header -<string.h> +<memory.h> declares one type, several functions, -several type-generic functions, +one type-generic function, and defines two macros useful for manipulating -arrays of character type and other objects treated as arrays of character type. -328) +AAA) The type is <b>size_t</b> and one of the macros is <b>NULL</b> (both described in 7.22). Various methods are used for determining the lengths of the arrays, but in all cases a -<b>char *</b> or <b>void *</b> argument points to the initial (lowest addressed) character of the array. If an array is accessed beyond the end of an object, the behavior is undefined. + +AAA) + See "future library directions" (7.35.<17+1>) @@ p2 +2 + The macro + __STDC_VERSION_MEMORY_H__ + is an integer constant expression + with a value equivalent to <tt>202ymmL</tt>. @@ ## Copy 7.28.1p3 as 7.<27+1>.1p3, verbatim. ## Copy 7.28.1p4 as 7.<27+1>.1p4, verbatim. +7.<27+1>.2 Copying functions ## Move 7.28.2.1 as 7.<27+1>.2.1 ("The memcpy function"). ## Move 7.28.2.2 as 7.<27+1>.2.2 ("The memccpy function"). ## Move 7.28.2.3 as 7.<27+1>.2.3 ("The memmove function"). +7.<27+1>.3 Comparison functions +7.<27+1>.3.1 General @@ Copy 7.28.4.1p1 as 7.<27+1>.3.1p1, then: The sign of a nonzero value returned by the comparison functions <b>memcmp</b> -, strcmp, and strncmp is determined by the sign of the difference between the values of the first pair of characters (both interpreted as unsigned char) that differ in the objects being compared. @@ ## Move 7.28.4.2 as 7.<27+1>.3.2 ("The memcmp function"). +7.<27+1>.4 Search functions +7.<27+1>.4.1 Introduction @@ Copy 7.28.5.1p1 as 7.<27+1>.4.1p1, then: The stateless search functions in this subclause (<b>memchr</b> -, strchr, strpbrk, strrchr, strstr ) are <i>generic functions</i>. These functions are generic in the qualification of the array to be searched and will return a result pointer to an element with the same qualification as the passed array. If the array to be searched is <b>const</b>-qualified, the result pointer will be to a <b>const</b>-qualified element. If the array to be searched is not <b>const</b>-qualified, -332) +BBB) the result pointer will be to an unqualified element. ## Copy footnote 332) as BBB), verbatim. @@ Copy 7.28.5.1p2 as 7.<27+1>.4.1p2, then: The external declarations of these generic functions have a concrete function type that returns a pointer to an unqualified element -( of type -<b>char</b> when specified as <bi>QChar</bi>, and <b>void</b> -when specified as <bi>QVoid</bi> -) , and accepts a pointer to a <b>const</b>-qualified array of the same type to search. This signature supports all correct uses. If the macro definition of any of these generic functions is suppressed to access an actual function, the external declaration with the corresponding concrete type is visible. -333) +CCC) @@ ## Copy footnote 333) as CCC), verbatim. ## Copy 7.28.5.1p3 as 7.<27+1>.4.1p3, verbatim. @@ ## Move 7.28.5.2 as 7.<27+1>.4.2 ("The memchr function"). +7.<27+1>.5 Miscellaneous functions ## Move 7.28.6.1 as 7.<27+1>.5.1 ("The memset function"). ## Move 7.28.6.2 as 7.<27+1>.5.2 ("The memset_explicit function"). 7.28 String handling <string.h> @@ New subsection after title +7.28.<0+1> General +1 + The header <string.h> + includes the headers <memory.h> and <nonstring.h>. @@ ## Move 7.28.1p2 as 7.28.<0+1>p2, verbatim. 7.28.1 String function conventions ## Delete p2 --moved elsewhere--. 7.28.2 Copying functions ## Delete 7.28.2.1 ("The memcpy function") --moved elsewhere--. ## Delete 7.28.2.2 ("The memccpy function") --moved elsewhere--. ## Delete 7.28.2.3 ("The memmove function") --moved elsewhere--. ## Delete 7.28.2.5 ("The strncpy function") --moved elsewhere--. ## Delete 7.28.2.7 ("The strndup function") --moved elsewhere--. 7.28.3 Concatenation functions @@ Title -Concatenation +Catenation @@ ## Delete 7.28.3.2 ("The strncat function") --moved elsewhere--. 7.28.4.1 Comparison functions :: General @@ p1 The sign of a nonzero value returned by the comparison functions +( -memcmp, strcmp -, and strncmp +) is determined by the sign of the difference between the values of the first pair of characters (both interpreted as unsigned char) that differ in the objects being compared. 7.28.4 Comparison functions ## Delete 7.28.4.2 ("The memcmp function") --moved elsewhere--. ## Delete 7.28.4.5 ("The strncmp function") --moved elsewhere--. 7.28.5.1 Search functions :: Introduction @@ p1 The stateless search functions in this subclause ( -memchr, <b>strchr</b>, <b>strpbrk</b>, <b>strrchr</b>, <b>strstr</b> ) are <i>generic functions</i>. These functions are generic in the qualification of the array to be searched and will return a result pointer to an element with the same qualification as the passed array. If the array to be searched is <b>const</b>-qualified, the result pointer will be to a <b>const</b>-qualified element. If the array to be searched is not <b>const</b>-qualified, 332) the result pointer will be to an unqualified element. @@ p2 The external declarations of these generic functions have a concrete function type that returns a pointer to an unqualified element -( of type <b>char</b> -when specified as <bi>QChar</bi>, -and <b>void</b> -when specified as <bi>QVoid</bi> -) , and accepts a pointer to a <b>const</b>-qualified array of the same type to search. This signature supports all correct uses. If the macro definition of any of these generic functions is suppressed to access an actual function, the external declaration with the corresponding concrete type is visible. 333) 7.28.5 Search functions ## Delete 7.28.5.2 ("The memchr function") --moved elsewhere--. 7.28.6 Miscellaneous functions ## Delete 7.28.6.1 ("The memset function") --moved elsewhere--. ## Delete 7.28.6.2 ("The memset_explicit function") --moved elsewhere--. ## Delete 7.28.6.5 ("The strnlen function") --moved elsewhere--. 7 Library @@ New subclause after 7.28 ("String handling <string.h>"). +7.<28+1> Nonstring handling <nonstring.h> +7.<28+1>.1 Nonstring function conventions @@ Copy 7.28.1p1 as 7.<28+1>.1p1, then: The header -<string.h> +<nonstring.h> declares one type, several functions, -several type-generic functions, and defines two macros useful for manipulating arrays of character type and other objects treated as arrays of character type. -328) +DDD) The type is <b>size_t</b> and one of the macros is <b>NULL</b> (both described in 7.22). Various methods are used for determining the lengths of the arrays, but in all cases a <b>char *</b> -or <b>void *</b> argument points to the initial (lowest addressed) character of the array. If an array is accessed beyond the end of an object, the behavior is undefined. + +DDD) + See "future library directions" (7.35.<18+1>) @@ p2 +2 + The macro + __STDC_VERSION_NONSTRING_H__ + is an integer constant expression + with a value equivalent to <tt>202ymmL</tt>. @@ ## Copy 7.28.1p3 as 7.<28+1>.1p3, verbatim. ## Copy 7.28.1p4 as 7.<28+1>.1p4, verbatim. +7.<28+1>.2 Copying functions ## Move 7.28.2.5 as 7.<28+1>.2.1 ("The strncpy function"). ## Move 7.28.2.7 as 7.<28+1>.2.2 ("The strndup function"). +7.<28+1>.3 Catenation functions ## Move 7.28.3.2 as 7.<28+1>.3.1 ("The strncat function"). +7.<28+1>.4 Comparison functions +7.<28+1>.4.1 General @@ Copy 7.28.4.1p1 as 7.<28+1>.4.1p1, then: The sign of a nonzero value returned by the comparison functions -memcmp, strcmp, and +( <b>strncmp</b> +) is determined by the sign of the difference between the values of the first pair of characters (both interpreted as unsigned char) that differ in the objects being compared. @@ ## Move 7.28.4.5 as 7.<28+1>.4.2 ("The strncmp function"). +7.<28+1>.5 Miscellaneous functions ## Move 7.28.6.5 as 7.<28+1>.5.1 ("The strnlen function"). ## I've kept plurals in sections that only have one function. ## This is because we may add functions in the future, so it's ## better to keep the wording generic to avoid having to update ## it needlessly.
signature.asc
(application/pgp-signature, 833 B)
-----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEES7Jt9u9GbmlWADAi64mZXMKQwqkFAmptBTgACgkQ64mZXMKQ wqlkwg//WZsPS2gCnwlzOoxXwb2DqT5AtJtHWKDXkcjHJHP2J6FTXwkbz0tEb1Da m+ZxVuwupA/lKweBj/UIJ+mkA4mnqQj03CPBb6V9eHGSoVUOir/haQwQ6JquQvvs 0jovp9GzuVIoekfvXZwP0jAov3mP5hR4vLh7o8DDAPNFOwsIV0MsC+pA4b++4d5F EDvnu4XiRYeNnS5oj2xdli8QvNgat0EAhHGAzSdLMjkBwA19Gu9+nkyhy1UW7K8m i7ZQPc4JnUqGJMMkHzc6wdfSAY0SpM58uWCCPHvntbIz0RYlK0bKaTmYN8tidgwW Jz5RQrAa4Uq3sb2+ZEkYdoaIwUBYQV4N+1RN64cQDKgY8rsb1QyNyzC5l6lmvnK/ Qenf5i61L27I0l2TAEpxFcwb9fI9ZGcbXhsUC4z9If/g7ZReJp55rJ2pFIVE3S/r nun1/VhF1+JFm2M3nHGTpxG0KGRvjflLyac5jEM1i4HJDQMHjAqzReEXyz3K19WJ caMW9oTPvXuk4PEmVhuyK/NW4KWOWNcsfk78gNEn3IXA6SWPS/oENw2p4TY+K8PN rGzYyET+1l8xuUBqmGTx3y5x1umV5jDU3U+hoUi6a4VVSvN8dt3RQ19dJ4ZXkpUI sMqYTMQnS4FI5atuWlasb0wO3wNDy1NZS73lLTMzLK6Oh9DLGWg= =j6My -----END PGP SIGNATURE-----