RE: ICU4C Initialization

Regarding Section 5, Part B:

Forgive me if I use terms that are classically C++ oriented. For those with their C hat on, please translate what I'm saying in your own mind to comparable idioms or methodologies.

To me this part of the proposal (Section 5, Part B) implies the use of double checked locking, and possibly in combination with the singleton design pattern. So it still introduces at least one static initializer for the lock to guard the internal singleton state, namely the pointer to the initialization object/functor. So unless I am missing something, which may possibly be so, it still introduces at least one static initialization - which is fine so long as you know the tricks to get it working on all supported platforms (esp. Mac).

If I could make a suggestion, since static initialization is really a compiler specific issue, go with the option in Section 5, Part A. If a developer using ICU wants to support thread safe static initialization of ICU, as detailed in Part B, they can do so (just as we have in MATLAB) by writing an init functor object, that calls u_init, and create a singleton of this object, to be called by each thread upon initialization (their thread factories can provide preconfigured threads if they so wish). So the point is that by going with the lowest common denominator, you can push the responsibility out to the ICU users. If the cross section of platforms users applications support handle static initialization easily, they can write it themselves. If the platform does not support static init, well they must call u_init from app main (essentially). If they are on a marginal platform, such as Mac (;^o), and if they want static init, then they will have to write some pretty sophisticat
 ed code to handle its completely non-standard conforming implementation for C++ static init.

My Vote: Option 5.A.

Bob

-----Original Message-----
From: Andy Heninger [mailto:[email protected]] 
Sent: Wednesday, August 20, 2003 8:45 PM
To: [email protected]
Subject: Re: ICU4C Initialization


Static Initialization of ICU, Take 2

I implemented the proposal from my earlier mail, included below, and it turned out to a bit problematic.  Briefly, the proposal was to make the use of u_init() mandatory in application using ICU, and do eliminate all use of
C++ static initialization in ICU.

The issue is that we have a number of tools and apps that were broken by doing this, even after adding u_init() to them, and based on this experience, I suspect that some significant number of other users of ICU would also have problems.

The scenario that goes wrong is this:  applications, for whatever reason, can't find the ICU data.  This causes u_init() to fail.  There wasn't a problem before, because some ICU services don't need data, and these were all that the application in question was using.  This situation was surprisingly common in our tools.

Problem #2:  u_init was not thread safe, and the usage model for multi-threaded applications could be awkward.  If multiple threads were started without u_init() being called in advance, it would be difficult for a thread to get ICU safely initialized without a lot of coordination with the other threads.

So here is another proposal to avoid breaking existing ICU users, while still eliminating C++ static initialization.

ICU Initialization, Proposal Number 2
-------------------------------------

1.  Apply cleverness to obtain a thread safe lazy initialization
    for ICU mutexes.  This turns out to be possible, although it
    requires different approaches between Windows and POSIX platforms.

2.  For all ICU services except Normalization and Character Properties,
    support lazy initialization.  Conversion, Collation, Locales,
    RBBI, and all the rest can used without first calling u_init(),
    although use of u_init() is encouraged.

3.  Normalization requires u_init().  The normalization API functions
    will attempt report back an error if used prior to calling u_init().
    (All of them take an error parameter).

    In some multi-processor, multi-threaded environments, the test
    for initialization wont be 100% reliable, and there is some chance
    of incorrect results.

    The problem with normalization is that many of the functions
    can be very quick, and to do a 100% reliable thread safe
    initialization check would be a significant relative cost.

4.  Character Properties.  The situation is similar to that with
    normalization, but there is no provision in the API for reporting
    errors.  We could do something along these lines:

    -  In debug builds of ICU, cause an assertion failure if ICU is
       not initialized.

    -  In release builds, do a u_init() if it looks like one hasn't
       yet been done.  Again, in some environments, the
       initialization check won't be 100% reliable


5.  Threads and u_init()

    Multi-threaded applications would have two ways in which u_init()
    could safely be used.

    a.  Call u_init() once, before creating any additional threads that
        will make use of ICU.  A call to u_init() will cover
        all thread created after that invocation.

        -- or  --

    b.  Call u_init() separately in each thread, before using other ICU
        functions.  u_init() will be thread safe, so no external
        synchronization among threads would be needed.

        Extra calls to u_init() would be harmless, so if a thread was
        unsure whether u_init() was needed, it could just do it again.
        (Harmless, except for the time required)

Opinions, anyone?

  -- Andy Heninger
     [email protected]


----- Original Message ----- 
From: "Andy Heninger" <[email protected]>
To: <[email protected]>
Sent: Tuesday, August 05, 2003 5:54 PM
Subject: ICU4C Initialization


> This is a proposal to change the way the ICU library is initialized.
>
> Here is some background, describing the way ICU works now, and why we 
> are thinking about changing it.
>
> Today (version 2.6), ICU is initialized as follows:
>
> 1.  At process static initialization time, mutexes are created
>     that are subsequently used during the lazy initialization
>     of the rest of ICU.
>
> 2.  An application may optionally call u_init() before
>     using other ICU functions.  This will do the mutex init
>     from step one on platforms where static init is not
>     reliable, and will load ICU's data.  Use of u_init() is
>     mandatory on certain multiprocessor systems.
>
> 3.  Any additional initialization for each ICU service is
>     done on demand, at the time that service is first used.
>
>
> There are two problems with this arrangement -
>
> 1.  For ICU 2.8, we will be introducing a mechanism to allow
>     applications to replace the memory heap and mutex
>     implementations that is used by ICU.  (I'll post a detailed
>     proposal for this shortly).
>
>     Static initialization will become a problem for environments
>     that want to use alternative mutexes or heaps, because
>     it (static init) happens before the application will have a
>     chance to specify the alternate heap/mutex implementations
>     to be used.
>
> 2.  In some configurations, thread safe initialization of ICU
>     already requires the use of u_init().  The potentially affected
>     ICU services are Normalization and Character Properties, and
>     affected machine configurations are multiple processor
>     systems using Alpha, Power 4 or Itanium CPUs, but not x86.
>
>     Macintoshes also require u_init(), because C++ static init
>     is not reliable.
>
>
> This proposal is to require the use of u_init() in all cases, and to
enforce
> this requirement by returning an error if an attempt is made to use 
> ICU services before u_init() has been called.  All C++ style static 
> initialization would be removed from the ICU library.
>
> This would be a breaking change for applications that do not already 
> call u_init(), so it's not something that we want to do without 
> careful
thought.
> Preventing potential threading problems is a major consideration; it's 
> better to catch these problems early, even if this causes a little 
> extra work for some developers who would not otherwise be affected.
>
>   -- Andy Heninger
>      [email protected]
> >

_______________________________________________
icu mailing list
[email protected] http://oss.software.ibm.com/developerworks/oss/mailman/listinfo/icu