Re: ICU4C Initialization

Static Initialization of ICU, Take 2

I implemented the proposal from my earlier mail, included below, and it
turned out to a bit problematic.  Briefly, the proposal was to make the use
of u_init() mandatory in application using ICU, and do eliminate all use of
C++ static initialization in ICU.

The issue is that we have a number of tools and apps that were broken by
doing this, even after adding u_init() to them, and based on this
experience, I suspect that some significant number of other users of ICU
would also have problems.

The scenario that goes wrong is this:  applications, for whatever reason,
can't find the ICU data.  This causes u_init() to fail.  There wasn't a
problem before, because some ICU services don't need data, and these were
all that the application in question was using.  This situation was
surprisingly common in our tools.

Problem #2:  u_init was not thread safe, and the usage model for
multi-threaded applications could be awkward.  If multiple threads were
started without u_init() being called in advance, it would be difficult for
a thread to get ICU safely initialized without a lot of coordination with
the other threads.

So here is another proposal to avoid breaking existing ICU users, while
still eliminating C++ static initialization.

ICU Initialization, Proposal Number 2
-------------------------------------

1.  Apply cleverness to obtain a thread safe lazy initialization
    for ICU mutexes.  This turns out to be possible, although it
    requires different approaches between Windows and POSIX platforms.

2.  For all ICU services except Normalization and Character Properties,
    support lazy initialization.  Conversion, Collation, Locales,
    RBBI, and all the rest can used without first calling u_init(),
    although use of u_init() is encouraged.

3.  Normalization requires u_init().  The normalization API functions
    will attempt report back an error if used prior to calling u_init().
    (All of them take an error parameter).

    In some multi-processor, multi-threaded environments, the test
    for initialization wont be 100% reliable, and there is some chance
    of incorrect results.

    The problem with normalization is that many of the functions
    can be very quick, and to do a 100% reliable thread safe
    initialization check would be a significant relative cost.

4.  Character Properties.  The situation is similar to that with
    normalization, but there is no provision in the API for reporting
    errors.  We could do something along these lines:

    -  In debug builds of ICU, cause an assertion failure if ICU is
       not initialized.

    -  In release builds, do a u_init() if it looks like one hasn't
       yet been done.  Again, in some environments, the
       initialization check won't be 100% reliable


5.  Threads and u_init()

    Multi-threaded applications would have two ways in which u_init()
    could safely be used.

    a.  Call u_init() once, before creating any additional threads that
        will make use of ICU.  A call to u_init() will cover
        all thread created after that invocation.

        -- or  --

    b.  Call u_init() separately in each thread, before using other ICU
        functions.  u_init() will be thread safe, so no external
        synchronization among threads would be needed.

        Extra calls to u_init() would be harmless, so if a thread was
        unsure whether u_init() was needed, it could just do it again.
        (Harmless, except for the time required)

Opinions, anyone?

  -- Andy Heninger
     [email protected]


----- Original Message ----- 
From: "Andy Heninger" <[email protected]>
To: <[email protected]>
Sent: Tuesday, August 05, 2003 5:54 PM
Subject: ICU4C Initialization


> This is a proposal to change the way the ICU library is initialized.
>
> Here is some background, describing the way ICU works now, and why we are
> thinking about changing it.
>
> Today (version 2.6), ICU is initialized as follows:
>
> 1.  At process static initialization time, mutexes are created
>     that are subsequently used during the lazy initialization
>     of the rest of ICU.
>
> 2.  An application may optionally call u_init() before
>     using other ICU functions.  This will do the mutex init
>     from step one on platforms where static init is not
>     reliable, and will load ICU's data.  Use of u_init() is
>     mandatory on certain multiprocessor systems.
>
> 3.  Any additional initialization for each ICU service is
>     done on demand, at the time that service is first used.
>
>
> There are two problems with this arrangement -
>
> 1.  For ICU 2.8, we will be introducing a mechanism to allow
>     applications to replace the memory heap and mutex
>     implementations that is used by ICU.  (I'll post a detailed
>     proposal for this shortly).
>
>     Static initialization will become a problem for environments
>     that want to use alternative mutexes or heaps, because
>     it (static init) happens before the application will have a
>     chance to specify the alternate heap/mutex implementations
>     to be used.
>
> 2.  In some configurations, thread safe initialization of ICU
>     already requires the use of u_init().  The potentially affected
>     ICU services are Normalization and Character Properties, and
>     affected machine configurations are multiple processor
>     systems using Alpha, Power 4 or Itanium CPUs, but not x86.
>
>     Macintoshes also require u_init(), because C++ static init
>     is not reliable.
>
>
> This proposal is to require the use of u_init() in all cases, and to
enforce
> this requirement by returning an error if an attempt is made to use ICU
> services before u_init() has been called.  All C++ style static
> initialization would be removed from the ICU library.
>
> This would be a breaking change for applications that do not already call
> u_init(), so it's not something that we want to do without careful
thought.
> Preventing potential threading problems is a major consideration; it's
> better to catch these problems early, even if this causes a little extra
> work for some developers who would not otherwise be affected.
>
>   -- Andy Heninger
>      [email protected]
> >