Re: standardizing linguistic encoding

Dear Eduard, [also addressing Philip and actually all... ]

It's probably my conditioning as a member of various standardization 
bodies that makes red lights flash in my head upon reading that you are 
"developing a standard"... :-) I believe that standards are better 
"developed" (or, more precisely, codified) on the basis of existing best 
practices or other existing standards. As it is, I can observe that your 
proposed encoding mixes up the level of tokens with the level of word 
forms (in ISO MAF terminology[1]), and while it can be suitable for your 
purposes, it is far from optimal in standardization terms.

[at this point, the camera pans out]

This year promises to be quite exciting for the TEI Linguistics SIG[2], 
given that:

(1) ISO LMF [3] is up for renewal and restructuring, and that several 
teams (among others, from ENeL, PARTHENOS, CLARIN, and LingSIG) are 
currently working on various modules for it,

(2) ISO Tiger [4] is nearing publication (as in: weeks rather than 
months) and opening a way for ISO TEIger, a TEI serialization of the ISO 
model for syntactic encoding,

(3) there is a rising push for streamlining inline linguistic markup, 
coming from, among others, Martin Mueller's Early Print Project, BBAW's 
existing practice (presented by Susanne Haaf at various TEI meetings), 
the Ancient Greek Dependency Treebank (represented in this mailing list 
by Giuseppe Celano, I believe), and now we learn of Philip Ströbel's 
project and yours. And there are others. A tiny reflex of that is 
contained at the LingSIG GitHub space [5], which is only meant as a 
_seed_ for collaborative effort rather than any personal statement.

Andreas Witt and I are thinking of how to address and channel this 
boiling mass of initiatives. One possibility could be to target the 
upcoming TEI Members Meeting[6] and have a focused pre-conference 
workshop designed to formulate a very precise and very concrete proposal 
for grammatical encoding synchronized across inline, standoff and 
dictionary markup, a proposal that we could submit to the TEI Technical 
Council at the end of the day. "The day" seems distant, but if we want 
to have a serious proposal at the end of it, work should start about now.

May I invite all interested parties to join the Linguistics SIG mailing 
list (by going to [7]) and GitHub space (by sending me, off-list, your 
github username), and to, well, have a go at it... :-)

Best regards,

   Piotr

[1]: 
http://www.iso.org/iso/home/store/catalogue_tc/catalogue_detail.htm?csnumber=51934
[2]: http://wiki.tei-c.org/index.php/SIG:TEI_for_Linguists
[3]: 
http://www.iso.org/iso/home/store/catalogue_tc/catalogue_detail.htm?csnumber=68516
[4]: 
http://www.iso.org/iso/home/store/catalogue_tc/catalogue_detail.htm?csnumber=62491
[5]: https://github.com/LingSIG/wordAttributes/wiki
[6]: http://members.tei-c.org/Events/meetings






On 02/01/17 16:51, Eduard Drenth wrote:
> Dear all,
>
> Here in Holland we are developing a standard to encode linguistic and
> lemma information for various word situations using TEI. We have been
> trying several solutions (tei:fs/tei:f, tei:interp, tei:span, ...) and
> finaly chose for TEI customization which gives us standard xsd
> validation, editor support and a simple focused solution. For linguistic
> terminology we use as much as possible http://universaldependencies.org/.
>
>
> We are curious as to what you think, see below for details. We hope this
> solution may be of use for those who want to encode linguistic
> information using TEI. Also this may help standardizing linguistic
> encoding in TEI.
>
>
> If this all is worthwhile I would like to donate/publish the solution
> somewhere.
>
>
> snippet customization:
>
>
>             <schemaSpec ident="tdb" docLang="en" prefix="tei_"
> xml:lang="en">
>
>                 ..
>
>                 ..
>
>                 <classSpec type="atts" ident="att.linguistics"
> module="analytics">
>
>                     <attList>
>                         <attDef ident="linguistics"
> ns="http://www.fryske-akademy.org/grammar/1.0">
>                             <desc>
>                                 documentation....
>                             </desc>
>                             <datatype maxOccurs="unbounded">
>                                 <dataRef key="teiata.enumerated"/>
>                             </datatype>
>                             <valList type="closed">
>                                 <valItem ident="Features.Abbr">
>                                     <desc>Boolean feature. Is this an
> abbreviation?</desc>
>                                 </valItem>
>                                 <valItem ident="Features.Poss">
>                                     <desc>Boolean feature of pronouns,
> determiners or adjectives. It tells whether the word is possessive.</desc>
>                                 </valItem>
>                                 <valItem ident="PronType.Prs">
>                                     <desc>personal pronoun or
> determiner</desc>
>                                 </valItem>
>
>                                  ..
>
>                                  ..
>
>
> example word encoding:
>
>
> <tei:w fa:linguistics="Pos.NOUN "
> lemmaRef="inprogress://lemmasystem/Hollands/frik/1"
> lemma="frik">Frik</tei:w>
>
>
> example split word encoding:
>
>
> <tei:w xml:id="staet-op-176" rendition="#split">staet</tei:w>
>
> <tei:w fa:linguistics="Pos.ADV "
> lemmaRef="inprogress://lemmasystem/Hollands/al/3"
> lemma="al">al</tei:w><tei:w>wringende</tei:w>
>
> <tei:w xml:id="staet-op-179" rendition="#split">op</tei:w>
>
> <tei:join result="w" scope="root" lemma="opstean" target="#staet-op-176
> #staet-op-179" lemmaRef="inprogress://lemmasystem/Hollands/opstean/1"
> fa:linguistics="th-si-pa Pos.VERB "/>
>
>
> example word consist of more lemma's (we don't use this yet....):
>
>
> <tei:choice>
>   <tei:orig>
>     <tei:w fa:linguistics=".....">aint</tei:w>
>   </tei:orig>
>   <tei:reg>
>     <tei:w lemma="be" fa:linguistics="...">am</tei:w>
>     <tei:w lemma="not" fa:linguistics="...">not</tei:w>
>   </tei:reg>
> </tei:choice>
>
>
> Bye,
>
>
> Eduard Drenth, Software Architekt
>
>
> [email protected]
>
>
> Doelestrjitte 8
>
> 8911 DX  Ljouwert
>
> +31 58 234 30 47
>
>
> gpg: https://sks-keyservers.net/pks/lookup?op=get&search=0x065EF82A1E02CC43
>

-- 
Piotr Bański, Ph.D.
Senior Researcher,
Institut für Deutsche Sprache,
R5 6-13
68-161 Mannheim, Germany