Re: Proposal for storing ID manifests in OpenEXR headers
Peter Hillman <[email protected]> Thu, 17 Aug 2017 17:27:45 +1200
| Newsgroups | gmane.comp.video.openexr.devel |
|---|---|
| Message-ID | <[email protected]> |
This is a multi-part message in MIME format. --===============7806844767310662831== Content-Type: multipart/alternative; boundary="------------29639FD799FDFE664365EAE2" Content-Language: en-US This is a multi-part message in MIME format. --------------29639FD799FDFE664365EAE2 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Hi Nick, Thanks for your comments. Yes, my section 1.2 is a little ambiguous: what I meant to say is that this proposal for an attribute is 'storage agnostic': the manifest should be compatible with most schemes for storing IDs in EXRs. It works both for schemes that use regular images and those that use deep. I pondered writing about UTF8 vs ASCII and decided not to for brevity. As it stands, you can use UTF8 to encode strings: reserving the ASCII '/' character (i.e. the byte code 0x2F) for hierarchy separation works with pure ASCII as well as UTF8. My understanding is that in UTF8 the byte range 0x00 to 0x7F always means ASCII characters; those bytes don't appear in multibyte character representations, as those all have their top bit set. There is something to be said for an attribute which defines the character encoding used for all 'human readable' attributes in an EXR, but that seems like a wider discussion. The semicolon is 'semi-special' under this scheme. Using it in strings shouldn't cause any errors. I wouldn't propose parsing strings concatenated like that in any software. It might be handy to display a concatenated name like "tunic;leather" and if there's a character which is rarely used elsewhere (escaped or otherwise) things will be clearer to read. If you are looking for a character to use for some special meaning with object names, try picking something other than semicolon. However, if you have a semicolon in your URI then don't bother changing it or escaping it. That said, the / character being a hierarchy separator will mean your URIs are likely to be treated as a hierarchy. That may well be desirable; if not you will need to encode it differently. Peter On 16/08/17 05:59, Nick Porcino wrote: > This looks reasonable to me. Almost every question I had was answered > by the document, > > One part that was unclear was 1.2, which talks about deep pixels and > ad hoc encodings in the same paragraph. I feel like I could read 1.2 > as deep pixels are, or are not, supported. I think they would be, but > would suggest breaking the discussion of deep and ad hoc apart. > > I deduce also that you are proposing ASCII encoding, although I think > UTF8 should be supported either as the designated encoding or as an > option. The reason for this is that in my own work I like to designate > ids with URI's and anticipate wanting to encode Japanese strings. Of > course I could introduce my own secondary mapping of exr id to URI at > the expense of extra work at my end, but I thought it's worth a bit of > discussion. > > Similarly, it's not clear to me that a separating character like > semicolon is necessary to be called out. If the intention is that > there will be a set of standard tools that can parse the assignments, > I can see the utility, but the counter examples like "tunic with > leather" suggests general parsing is perhaps not anticipated. > > If semicolon is to be somehow special then I suggest that the manner > of escaping semicolon also be described. Could be as simple as ;; or > \; means a literal semicolon, not a separator. > > Happy to see new thoughts around EXR, > > - Nick p > > ------------------------------------------------------------------------ > *From:* Openexr-devel > <[email protected]> on behalf of > Peter Hillman <[email protected]> > *Sent:* Monday, August 14, 2017 8:23:51 PM > *Cc:* [email protected] > *Subject:* [Openexr-devel] Proposal for storing ID manifests in > OpenEXR headers > Hi all, > > As Deke mentioned earlier, At Siggraph I presented a proposal for an > OpenEXR attribute to store ID manifests (i.e. a table that maps > between text string names of entities and the numerical values encoded > in the EXR image) Attached is an updated draft giving more details of > this proposal. There are a few modifications from the earlier draft > following suggestions from those attending Siggraph: many thanks to > all those who commented. > > This is /not/ a proposed complete single standard for storing IDs > within OpenEXRs. There are reasons to choose one storage scheme over > another depending on circumstances, suggesting that it is worthwhile > to support different encoding schemes. The requirement for an ID > manifest is a common requirement amongst those schemes. This document > merely defines a standard attribute for encoding the manifest and for > allowing it to be compressed. This should allow for a standard set of > tools for reading, writing and examining the attribute. > > The intention is that I will implement this attribute in the next few > months - certainly before the end of the year. Feel free to comment > on the standard before then! > > Peter > > --------------29639FD799FDFE664365EAE2 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: 8bit <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=windows-1252"> </head> <body text="#000000" bgcolor="#FFFFFF"> <p>Hi Nick,</p> <p>Thanks for your comments.</p> <p>Yes, my section 1.2 is a little ambiguous: what I meant to say is that this proposal for an attribute is 'storage agnostic': the manifest should be compatible with most schemes for storing IDs in EXRs. It works both for schemes that use regular images and those that use deep. <br> </p> <p>I pondered writing about UTF8 vs ASCII and decided not to for brevity. As it stands, you can use UTF8 to encode strings: reserving the ASCII '/' character (i.e. the byte code 0x2F) for hierarchy separation works with pure ASCII as well as UTF8. My understanding is that in UTF8 the byte range 0x00 to 0x7F always means ASCII characters; those bytes don't appear in multibyte character representations, as those all have their top bit set. There is something to be said for an attribute which defines the character encoding used for all 'human readable' attributes in an EXR, but that seems like a wider discussion.<br> </p> <p>The semicolon is 'semi-special' under this scheme. Using it in strings shouldn't cause any errors. I wouldn't propose parsing strings concatenated like that in any software. It might be handy to display a concatenated name like "tunic;leather" and if there's a character which is rarely used elsewhere (escaped or otherwise) things will be clearer to read. If you are looking for a character to use for some special meaning with object names, try picking something other than semicolon. However, if you have a semicolon in your URI then don't bother changing it or escaping it. That said, the / character being a hierarchy separator will mean your URIs are likely to be treated as a hierarchy. That may well be desirable; if not you will need to encode it differently.<br> </p> <p>Peter<br> </p> <div class="moz-cite-prefix">On 16/08/17 05:59, Nick Porcino wrote:<br> </div> <blockquote type="cite" cite="mid:DM5PR1601MB13564E83D120816F37E68F58A48D0@DM5PR1601MB1356.namprd16.prod.outlook.com"> <meta http-equiv="Content-Type" content="text/html; charset=windows-1252"> This looks reasonable to me. Almost every question I had was answered by the document, <br> <br> One part that was unclear was 1.2, which talks about deep pixels and ad hoc encodings in the same paragraph. I feel like I could read 1.2 as deep pixels are, or are not, supported. I think they would be, but would suggest breaking the discussion of deep and ad hoc apart.<br> <br> I deduce also that you are proposing ASCII encoding, although I think UTF8 should be supported either as the designated encoding or as an option. The reason for this is that in my own work I like to designate ids with URI's and anticipate wanting to encode Japanese strings. Of course I could introduce my own secondary mapping of exr id to URI at the expense of extra work at my end, but I thought it's worth a bit of discussion.<br> <br> Similarly, it's not clear to me that a separating character like semicolon is necessary to be called out. If the intention is that there will be a set of standard tools that can parse the assignments, I can see the utility, but the counter examples like "tunic with leather" suggests general parsing is perhaps not anticipated.<br> <br> If semicolon is to be somehow special then I suggest that the manner of escaping semicolon also be described. Could be as simple as ;; or \; means a literal semicolon, not a separator.<br> <br> Happy to see new thoughts around EXR,<br> <br> - Nick p<br> <br> <hr style="display:inline-block;width:98%" tabindex="-1"> <div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" color="#000000" face="Calibri, sans-serif"><b>From:</b> Openexr-devel <a class="moz-txt-link-rfc2396E" href="mailto:[email protected]"><[email protected]></a> on behalf of Peter Hillman <a class="moz-txt-link-rfc2396E" href="mailto:[email protected]"><[email protected]></a><br> <b>Sent:</b> Monday, August 14, 2017 8:23:51 PM<br> <b>Cc:</b> <a class="moz-txt-link-abbreviated" href="mailto:[email protected]">[email protected]</a><br> <b>Subject:</b> [Openexr-devel] Proposal for storing ID manifests in OpenEXR headers</font> <div> </div> </div> <div>Hi all,<br> <br> As Deke mentioned earlier, At Siggraph I presented a proposal for an OpenEXR attribute to store ID manifests (i.e. a table that maps between text string names of entities and the numerical values encoded in the EXR image) Attached is an updated draft giving more details of this proposal. There are a few modifications from the earlier draft following suggestions from those attending Siggraph: many thanks to all those who commented.<br> <br> This is <i>not</i> a proposed complete single standard for storing IDs within OpenEXRs. There are reasons to choose one storage scheme over another depending on circumstances, suggesting that it is worthwhile to support different encoding schemes. The requirement for an ID manifest is a common requirement amongst those schemes. This document merely defines a standard attribute for encoding the manifest and for allowing it to be compressed. This should allow for a standard set of tools for reading, writing and examining the attribute. <br> <br> The intention is that I will implement this attribute in the next few months - certainly before the end of the year. Feel free to comment on the standard before then!<br> <br> Peter<br> <br> <br> </div> </blockquote> <br> </body> </html> --------------29639FD799FDFE664365EAE2-- --===============7806844767310662831== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Openexr-devel mailing list [email protected] https://lists.nongnu.org/mailman/listinfo/openexr-devel --===============7806844767310662831==--