Re: language variant subtag "pinyin" modification request

Hugh Paterson wrote:

> I'm interested in some clarification on the statement below:
>
>> It also suggests using invalid tags in the Japan section, where it
>> makes the bizarre statement that "BCP 47 makes no difference between
>> '-' and '_'". Be careful when citing this project.
>
> I'm wondering if the source of confusion at the OSM project might be
> CLDR. I notice that the documentation at CLDR for suggested tags
> includes underscores. See the documentation at [1].
>
> For my own understanding, am I correct to understand that CLDR tags
> are not BCP47 tags? or put another way what is the relationship
> between CLDR tags (approved ones) and BCP47 tags (approved ones).
>
> [1]: 
> http://www.unicode.org/cldr/charts/latest/supplemental/likely_subtags.html

CLDR defines its own extensions of the BCP 47 syntax. Here is the 
relevant passage from UTS #35, which defines the syntax for identifiers 
(and much more) in CLDR:

> 3.3 BCP 47 Conformance
>
> Unicode language and locale identifiers inherit the design and the
> repertoire of subtags from [BCP47] Language Tags. There are some
> extensions and restrictions made for the use of the Unicode locale
> identifier in CLDR:
>
> [...]
>
> It allows certain syntax for backwards compatibility (not BCP 47-
> compatible):
>
> The "_" character for field separator characters, as well as the "-"
> used in [BCP47] (however, the canonical form is with "-")

So underscores are not allowed in BCP 47 tags, but they are allowed in 
CLDR identifiers.

As a side note, CLDR "likely subtags" are not necessary the "suggested 
tags" for language tagging. As Section 4.3 of UTS #35 notes, the purpose 
of these is to indicate the most likely language, script, and/or region 
subtag, given one or two of the others. They do not supersede the 
general BCP 47 advice to omit needless subtags.

For example, although likelySubtags.xml defines this:

<likelySubtag from="ja" to="ja_Jpan_JP"/>

there is no reason in normal language tagging to use either 'Jpan' or 
'JP' in this tag. It is just telling you what the most likely script and 
region subtags for Japanese would be.

CLDR is an important user of BCP 47, but its syntax and rules are its 
own.

--
Doug Ewell | Thornton, CO, US | ewellic.org