ICU4C proposal: Formalizing the ustdio format specifiers

Deadline for comments: July 16, 2003


Introduction

For ICU 2.8 we are making ustdio a fully supported library.  Basically, 
this means that I will be fixing many broken things in the ustdio library. 
 Part of this fixing process is to formalize how the format specification 
string works for fscanf, fprintf and related functions.

As a reminder, the ustdio library has been marked as draft for a while, 
which means that the API may change at any time.  We have also stated that 
ustdio is a broken unsupported library for a long time.  I'd like to make 
the ustdio library more compatible with the ANSI C stdio format 
specification.  Backwards compatibility with the old ustdio is not a goal 
of this proposal, since that would involve keeping many bugs from the old 
ustdio around for eternity.

This is only a partial proposal.  A full proposal would be many pages 
long, and most people wouldn't read it.  So I'm only sending out a "small" 
proposal for part of my ustdio work at this time so that it is easier to 
read (hopefully).  More proposals will appear later on, like on how some 
of the function APIs will look.


Actual Proposal

Here are the format specifications that I plan to use in the new supported 
ustdio library.  These formats apply to u_fscanf, u_fprintf, u_sscanf, 
u_sprintf and all related functions in ustdio.h.

printf
fmt type        Comment
%E  double      Scientific with an uppercase exponent
%e  double      Scientific with a lowercase exponent
%G  double      Use %E or %f for best format
%g  double      Use %e or %f for best format
%f  double      Simple floating point without the exponent
%X  int32_t     ustdio special uppercase hex radix formatting
%x  int32_t     ustdio special lowercase hex radix formatting
%d  int32_t     Decimal format
%i  int32_t     Same as %d
%n  int32_t     count (write the number of chars written)
%o  int32_t     octal ustdio special octal radix formatting
%u  uint32_t    Decimal format
%p  void *      Prints the pointer value
%s  char *      Use default converter or specified converter from fopen
%hs char *      Use invariant converter
%ls N/A         (Unimplemented) Reserved for future implementation
%c  char        Use default converter or specified converter from fopen
%hc char        Use invariant converter
%lc N/A         (Unimplemented) Reserved for future implementation
%S  UChar *     Null terminated UTF-16 string
%hS char *      Null terminated UTF-8 string
%lS UChar32 *   Null terminated UTF-32 string
%C  UChar       16-bit Unicode code unit
%hC char        8-bit Unicode code unit
%lC UChar32     32-bit Unicode code unit
%%  N/A         Show a percent sign

scanf differences from printf
%[] UChar *     Scanset needs to be replaced with ICU UnicodeSet.
                Doesn't contain s, S or any other string format.

Format modifiers
%l    int64_t   long format for %d, %i, %o, %x (usually this means long 
int)
%h    int16_t   short format for %d, %i, %o, %x (usually this means short 
int)
%l    uint64_t  long format for %u (usually this means long int)
%h    uint16_t  short format for %u (usually this means short int)
%-    N/A       Left justify
%+    N/A       Always show the plus or minus sign. Needs data for plus 
sign.
%     N/A       Instead of a "+" output a blank character for positive 
numbers.
%#    N/A       Precede octal value with 0, hex with 0x and show the 
decimal point for floats.
%num  N/A       Width of input/output. num is an actual number from 0 to 
some large number.
%.num N/A       Significant digits precision. num is an actual number from 
0
                to some large number. Currently can only specify precision
                before or after decimal, and not total precision.

printf modifier
%*  int32_t     Next argument after this one specifies the width (need to 
implement)

scanf modifier
%*  N/A         This field is scanned, but not stored (need to implement)

Special extensions reserved for future consideration
%P  double      Percent format
%V  double      Spellout format
%b  char *      HP-UX and glibc extension which allows you to unescape a 
string
%B  UChar *     Similar to %b
% $ N/A         positional formatting modifier like in MessageFormat from 
the "Single Unix Specification".

Formats removed due to other formats or functions.
%T  UDate       Time. Replaced by strftime/strptime.
%D  UDate       Date. Replaced by strftime/strptime.
%K  UChar       Source or target is UChar. Replaced by %C.
%U  UChar *     Source or target is UChar *. Replaced by %S.
%M  double      Currency format. Replaced by strfmon. strpmon may be 
implemented in the future.



Other related changes

Functions removed
u_getcx()       %b will replace this function in the future

The char * format specification will now be converted with u_charsToUChars 
instead of the default converter.  If you want to display non-ASCII 
characters, you need to use the functions that use a const UChar * format 
argument.  This is being done because:

1) It's much faster
2) The format specification argument is usually a static char *, but the 
default converter may convert the string differently depending on the 
platform ICU is being run on.  This is generally bad, and that's why the 
UChar * format argument should be used for internationalized text instead 
of the API that takes a char * formatting argument.  The %s argument will 
still use the default converter since the source of the string is much 
more likely to come from an outside source.



Thank you for taking time to read this proposal.

George Rhoten
IBM Globalization Center of Competency/ICU  San Jose, CA, USA