RE: ICU4C proposal: Formalizing the ustdio format specifiers

Great. I look forward to the changes. Esp support for %ld.

A couple questions though...

############################################################################
##
1.

From what I am reading in the C99 spec before me (Ch 7.19.6.1 article 4 and
Ch 7.19.6.2 article 11):

, the conversion specifier must be present
, the length modifer is optional
, behavior of a specification using a length modifier by itself is undefined

Does your spec suggest that '%l' is equivalent to '%ld' ? If so, please no.

Also, the spec says...

", l (ell) Specifies that a following d, i, o, u, x, or X conversion
specifier
applies to a long int or unsigned long int argument; that a following n
conversion specifier applies to a pointer to a long int argument; that a
following c conversion specifier applies to a wint_t argument; that a
following s conversion specifier applies to a pointer to a wchar_t
argument; or has no effect on a following a, A, e, E, f, F, g, or G
conversion
specifier.

", ll (ell-ell) Specifies that a following d, i, o, u, x, or X conversion
 specifier applies to a long long int or unsigned long long int argument;
or that a following n conversion specifier applies to a pointer to a long
long int
argument."

It would seem to me that, apart from those outlier compilers that do not
support 'long int' or 'long long int', that the 'l' and 'll' should behave
as though the modifer indicates that the subsequent argument is at most the
platform width of 'long int' for 'l', and the platform width of 'long long
int'
for 'll'. Taking a poll on five OSs, this means these would map to:

type / sizeof   windows-ia32-vc7  linux-ia32  solaris-64  macosx  hpux-11
--------------------------------------------------------------------------
'long long int'   unsupported          8         8           8        8
'long int'             4               4         4           4        4

For win32 it would be reasonable to say that 'll' regards 64-bit types.

I think it would be reasonable to say from evidence that %ld is used for
int32_t,
and %lld is used for int64_t. Similarly for corresponding unsigned
quantities as well,
%lu and %llu. In fact, if you use the '%ld' format on hpux for a 64 bit
type,
and a value of INT64_MAX, you get an error, which seems to further support
my
argument.

 printf("sizeof long int: %lld\n", INT64_MAX); // yields 9223372036854775807
 printf("sizeof long int: %ld\n", INT64_MAX); // yields 1001; an error of
course

Other platforms that support the spec for 'll' will yield similar errors.

So in summary, %ld is _not_ for int64_t, nor is %lu for uint64_t according
to the spec,
or according to practice. Please do not introduce this error to the ICU api.

############################################################################
##
2.

Also, regarding the 'h' modifier and its use with regards to unicode
strings,
another alternative would be:

> %S  UChar32 *  Null terminated UTF-16 string
> %hS UChar *    Null terminated UTF-16 string
> %hhS char *    Null terminated UTF-32 string

and,

> %C UChar32     32-bit Unicode code unit
> %hC  UChar     16-bit Unicode code unit
> %hhC char      8-bit Unicode code unit

The point is that the conversion type for an 'h' modifier is presumed to be
promoted to sizeof int. The int type is more naturally associated to
UChar32 than UChar16. So one could argue that the above is more
consistent with the intent of the C99 Spec on the basis that 'h' is
meant to demote a type from an int to a type of short, and 'hh' is meant
to demote an int to a char. Your use of 'h' is inconsistent with the
spec as it demotes a short to a char. See Ch 7.24.2.1 article 7.

############################################################################
##
3.

> %ls N/A         (Unimplemented) Reserved for future implementation

Actually %ls is already taken by the C99 Spec, and refers to a wchar_t *.
So please consider dropping this from your spec, but support it according
to platform/compiler conventions, and the C99 spec.

> %lc N/A         (Unimplemented) Reserved for future implementation

And in the spec this refers to a wint_t. Same as above.

############################################################################
##
4.

A nifty thing we use regularly in MATLAB is function holes. It essentially
does a callback to a function that is allowed to put N characters into
the output buffer. For us it is represented as a naked %U, so leaving out
lots of details:

	switch (conversion specifier hole type) {
	   case FUNCTION_HOLE_TYPE:
		{
		    va_list   hole_ap = msg->hole_infos[holeIdx].hole.hole_ap;
		    fn_fmtfcn fmtfcn  = va_arg(hole_ap, fn_fmtfcn);
		    num_chars_written = (fmtfcn)(putsn, dest_buf_or_file_ptr, &hole_ap);
		    break;
		}
	    }

	typedef int (*fn_fmtfcn)(  /* returns the number of chars putsn'd after
processing*/
		fn_putsn putsn,
		void *x,             /* destination info, e.g., FILE *, or &buffer */
		va_list *ap);        /* var args list to custom format specifier, on
return
                          this is advanced past the custom format varargs */

Would it be possible to augment the icu api to support this sort of
callback?

-Bob

> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]]On Behalf Of George
> Rhoten
> Sent: Wednesday, July 09, 2003 7:05 PM
> To: [email protected]
> Subject: ICU4C proposal: Formalizing the ustdio format specifiers
>
>
> Deadline for comments: July 16, 2003
>
>
> Introduction
>
> For ICU 2.8 we are making ustdio a fully supported library.  Basically,
> this means that I will be fixing many broken things in the ustdio
> library.
>  Part of this fixing process is to formalize how the format specification
> string works for fscanf, fprintf and related functions.
>
> As a reminder, the ustdio library has been marked as draft for a while,
> which means that the API may change at any time.  We have also
> stated that
> ustdio is a broken unsupported library for a long time.  I'd like to make
> the ustdio library more compatible with the ANSI C stdio format
> specification.  Backwards compatibility with the old ustdio is not a goal
> of this proposal, since that would involve keeping many bugs from the old
> ustdio around for eternity.
>
> This is only a partial proposal.  A full proposal would be many pages
> long, and most people wouldn't read it.  So I'm only sending out
> a "small"
> proposal for part of my ustdio work at this time so that it is easier to
> read (hopefully).  More proposals will appear later on, like on how some
> of the function APIs will look.
>
>
> Actual Proposal
>
> Here are the format specifications that I plan to use in the new
> supported
> ustdio library.  These formats apply to u_fscanf, u_fprintf, u_sscanf,
> u_sprintf and all related functions in ustdio.h.
>
> printf
> fmt type        Comment
> %E  double      Scientific with an uppercase exponent
> %e  double      Scientific with a lowercase exponent
> %G  double      Use %E or %f for best format
> %g  double      Use %e or %f for best format
> %f  double      Simple floating point without the exponent
> %X  int32_t     ustdio special uppercase hex radix formatting
> %x  int32_t     ustdio special lowercase hex radix formatting
> %d  int32_t     Decimal format
> %i  int32_t     Same as %d
> %n  int32_t     count (write the number of chars written)
> %o  int32_t     octal ustdio special octal radix formatting
> %u  uint32_t    Decimal format
> %p  void *      Prints the pointer value
> %s  char *      Use default converter or specified converter from fopen
> %hs char *      Use invariant converter
> %ls N/A         (Unimplemented) Reserved for future implementation
> %c  char        Use default converter or specified converter from fopen
> %hc char        Use invariant converter
> %lc N/A         (Unimplemented) Reserved for future implementation
> %S  UChar *     Null terminated UTF-16 string
> %hS char *      Null terminated UTF-8 string
> %lS UChar32 *   Null terminated UTF-32 string
> %C  UChar       16-bit Unicode code unit
> %hC char        8-bit Unicode code unit
> %lC UChar32     32-bit Unicode code unit
> %%  N/A         Show a percent sign
>
> scanf differences from printf
> %[] UChar *     Scanset needs to be replaced with ICU UnicodeSet.
>                 Doesn't contain s, S or any other string format.
>
> Format modifiers
> %l    int64_t   long format for %d, %i, %o, %x (usually this means long
> int)
> %h    int16_t   short format for %d, %i, %o, %x (usually this means short
> int)
> %l    uint64_t  long format for %u (usually this means long int)
> %h    uint16_t  short format for %u (usually this means short int)
> %-    N/A       Left justify
> %+    N/A       Always show the plus or minus sign. Needs data for plus
> sign.
> %     N/A       Instead of a "+" output a blank character for positive
> numbers.
> %#    N/A       Precede octal value with 0, hex with 0x and show the
> decimal point for floats.
> %num  N/A       Width of input/output. num is an actual number from 0 to
> some large number.
> %.num N/A       Significant digits precision. num is an actual
> number from
> 0
>                 to some large number. Currently can only specify precision
>                 before or after decimal, and not total precision.
>
> printf modifier
> %*  int32_t     Next argument after this one specifies the width (need to
> implement)
>
> scanf modifier
> %*  N/A         This field is scanned, but not stored (need to implement)
>
> Special extensions reserved for future consideration
> %P  double      Percent format
> %V  double      Spellout format
> %b  char *      HP-UX and glibc extension which allows you to unescape a
> string
> %B  UChar *     Similar to %b
> % $ N/A         positional formatting modifier like in MessageFormat from
> the "Single Unix Specification".
>
> Formats removed due to other formats or functions.
> %T  UDate       Time. Replaced by strftime/strptime.
> %D  UDate       Date. Replaced by strftime/strptime.
> %K  UChar       Source or target is UChar. Replaced by %C.
> %U  UChar *     Source or target is UChar *. Replaced by %S.
> %M  double      Currency format. Replaced by strfmon. strpmon may be
> implemented in the future.
>
>
>
> Other related changes
>
> Functions removed
> u_getcx()       %b will replace this function in the future
>
> The char * format specification will now be converted with
> u_charsToUChars
> instead of the default converter.  If you want to display non-ASCII
> characters, you need to use the functions that use a const UChar * format
> argument.  This is being done because:
>
> 1) It's much faster
> 2) The format specification argument is usually a static char *, but the
> default converter may convert the string differently depending on the
> platform ICU is being run on.  This is generally bad, and that's why the
> UChar * format argument should be used for internationalized text instead
> of the API that takes a char * formatting argument.  The %s argument will
> still use the default converter since the source of the string is much
> more likely to come from an outside source.
>
>
>
> Thank you for taking time to read this proposal.
>
> George Rhoten
> IBM Globalization Center of Competency/ICU  San Jose, CA, USA
> _______________________________________________
> icu mailing list
> [email protected]
> http://oss.software.ibm.com/developerworks/oss/mailman/listinfo/icu