Re: encoding and content in YML files

Thanks Ben.  We’re missing the point again, and it’s me. I am trying to get to the bottom of something and I can not explain it.  Darn it.  Let me try it again...

I provided a YML file, UTF-8 encoded, to our software team.  They said to me, we need the special characters to be unicode encoded and they gave me examples by saying instead of this, Ă we need \u0102.  So, I said the YML site says that you should be able to see those characters ok.  And they said, “No, I need the Unicode code representation as in \u0102.  So, I asked this team about it, and I am still receiving help on various UTF-XX encoding. :-)  No problem, I am getting smarter with this.  And I think, that from all you are all saying, I should be able to provide an UTF-8 encoded file and the programmers should be able to handle it.  Thanks for your help all.

josé

From: Oren Ben-Kiki 
Sent: Thursday, October 15, 2015 3:55 PM
To: jose isaias cabrera 
Cc: Yaml Core 
Subject: Re: [Yaml-core] encoding and content in YML files

Doesn't work that way :-) 

"Code point" is a "platonic ideal". \u1234, UTF8, UTF16LE, UTF16BE, UTF32LE, UTF32BE, etc. are all different ways to encode it.

Think of the number three. You can't put the number three into a file. You can put the byte 0b00000011 into a file, encoding it as a 1-byte integer (uint8_t). Or you can write the ascii string 't' 'h' 'r' 'e' 'e' into a file. You you could put the ascii character '3' into a file. Or any of a zillion other ways. All these are _encodings_. The number 3 itself is neither of them. It is a platonic ideal (the successor of the successor of the successor of the zero element, if you go by Peano's axioms - and the previous sentence is yet another "encoding").

A "code point" is like that. You can only put an _encoding_ of a code point into a file.


On Thu, Oct 15, 2015 at 10:45 PM, jose isaias cabrera <[email protected]> wrote:


  Sorry for the late reply:
  This is what I mean for actual characters:
  ጒፄ䢘䔲

  unicode code points:
  \u1312\u1344\u4898\u4532


  From: Oren Ben-Kiki 
  Sent: Thursday, October 08, 2015 2:36 AM
  To: jose isaias cabrera 
  Cc: Yaml Core 
  Subject: Re: [Yaml-core] encoding and content in YML files

  Not sure what you mean by "the actual characters vs. the Unicode code points". The _only_ thing a physical file can contain is some _encoding_ of "the actual characters". If someone sees garbage displaying the file, probably his font doesn't support the used characters, or his system doesn't support the encoding, or something like that.

  On Thu, Oct 8, 2015 at 12:14 AM, jose isaias cabrera <[email protected]> wrote:


    thanks for the reply...

    I know what Unicode and the various UTF-XX formats are.  So, you're missing the "point" ;-) he he he.  Thanks for pointing me to the site referenced below.  That helped me.  Unicode code points are not Unicode display characters, such as ε, φ, é, ñ, α, etc.  they are displayed as codes, such as \u039A, \u03BB, \u03B5, \u03AF, etc. So, in other words, what YAML files need are Unicode code points, \uxxxx, instead of the actual characters. So, why can't YML take the actual characters vs. the Unicode code points?  The reason why I know this is because I gave someone a YML UTF-8 encoded with characters set and the person told me that they appeared as garbage.  Anyway, my personal opinion is that YML suffers by not allowing the characters.

    thanks.

      ----- Original Message ----- 
      From: Oren Ben-Kiki 
      To: jose isaias cabrera 
      Cc: Yaml Core 
      Sent: Wednesday, October 07, 2015 12:24 AM
      Subject: Re: [Yaml-core] encoding and content in YML files

      Valid UTF-8 encoding is an encoding of "Unicode code points" :-) 

      "Unicode code points" is a technical term - http://gedcom-parse.sourceforge.net/doc/encoding.html is an OK explanation. Unicode code points (or "Unicode characters", if you want to be less formal) are in the range 0 to 10FFFF. Obviously you can't encode one such beast in one byte. Hence, Unicode supports multiple ways of expressing these code points as a sequence of bytes, UTF-8 being one such way to encode them, as are the two UTF-16 (big and small endian) encodings.

      On Wed, Oct 7, 2015 at 12:48 AM, jose isaias cabrera <[email protected]> wrote:


        Greetings!

        I have searched all over the internet for answers, and even on the YAML documentation, c5.2, it's not clear to me. The statement,

        All characters mentioned in this specification are Unicode code points.

        in here,

        http://www.yaml.org/spec/1.2/spec.html#id2771184

        is confusing.  Maybe is because I don't understand what "Unicode code points" means.  So, does this mean that all .yml files can not have characters such as é or ñ, etc.?  I would expect that a valid UTF encoding on the .yml file should be ok to have these unicode actual characters.  Thanks for your support.

        josé

        ------------------------------------------------------------------------------

        _______________________________________________
        Yaml-core mailing list
        [email protected]
        https://lists.sourceforge.net/lists/listinfo/yaml-core

------------------------------------------------------------------------------

_______________________________________________
Yaml-core mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/yaml-core