Re: standardizing linguistic encoding

Hi,

I am an African-Based linguist, and I have been following much of the
discussions going on over TEI mailing list.

I am not a TEI practitioner per se, but I am aware of the many application
scenarios of this technology, including text corpora building.

A couple of months ago, I sent an e-mail around via TEI mailing list asking
whether anyone knew of any TEI based/inspired framework for the encoding of
prosodic phenomena such as tones, especially in African tone languages. I
got one or two responses from members. Unfortunately these responses did
not address my specific concern.

I wish to join on-going discussions about 'standardizing linguistic
encoding', to bring to the fore of TEI standards development, the issue of
"tone encoding".

For the sake of clarification and given that not every one is necessarily
an expert in tone languages, let me explain by examples what tone is in
African tone languages.

Given the followings tokens from Basaa, a bantu language spoken in Cameroon:

(1) hól : to sharpen

(2) hòl : to pay the dawry

(3) hôl (as in *á hôl*): let him sharpen

(4) hŏl : pay the dawry! (imperative)

In (1) through (4), the difference in meaning of these words is attributed
to the difference in relative pitch level of the syllable: "high" in (1),
"low" in (2), contour or two-level "low-high" in (3), contour or two-level
"low-high" in (4).

While the semantics associated with tone levels in (1) and (2) is lexically
encoded, the ones in (3) and (4) are complemented with grammatical
information, namely hortative in (3) and imperative in (4), thus resulting
in complex (contour) tone shapes in writing.

Tone representation in the above examples is graphical, and is meant to
simply anchor pitch melody; this form of representation does not inform
much about the semantics associated with a specific pitch level in and
accross words. This is so mostly because pitch 'labels' (high, low,
low-high, high-low) do not encode persistent meaning, but may instead
trigger each and array of grammatical information such as tense, mood,
aspect, negation, ect., depending on the context.

I personally believe that for better processeability and representation of
textual information in tone langues, there is need for developping
unambiguous encoding framework devoid of graphical representation of tones,
and I believe TEI to be one possible response to this.

Because TEI is an open standard which is meant to be tailored to the
specific needs of users, I think it is our responsiblity as Africanists and
Bantuists, to raise TEI community's awarness about accounting for the
specificities of the languages we are working on, when it comes to
standardizing linguistic encoding.

Best

Emmanuel Ngué Um
Language Archivist for ALORA

2017-02-02 14:03 GMT+01:00 Eduard Drenth <[email protected]>:

> Thanks for your response! Standard in my case means practical, usable way
> for encoding linguistic information in corpora using TEI.
>
> Indeed the theme is covered by https://github.com/LingSIG/
> wordAttributes/wiki. Good to know of this http://wiki.tei-c.org/index.
> php/SIG:TEI_for_Linguists as well.
>
> We choose to continue along the choosen path, it doesn't deviate too much
> from uncustomized TEI, offers good support for editing and querying,
> satisfies our linguists, adheres to http://universaldependencies.org and
> is easy to convert to the 'real standard' when it is released.
>
> Perhaps our approach can be useful input for https://github.com/LingSIG/
> wordAttributes, it is the result of quite extensive testing and
> discussing.
>
> Eduard Drenth, Software Architekt
>
> [email protected]
>
> Doelestrjitte 8
> 8911 DX  Ljouwert
> +31 58 234 30 47
>
> gpg: https://sks-keyservers.net/pks/lookup?op=get&search=
> 0x065EF82A1E02CC43
>
> ________________________________________
> From: Piotr Bański <[email protected]>
> Sent: Thursday, February 2, 2017 12:24 PM
> To: Eduard Drenth; [email protected]; Phillip Ströbel
> Subject: Re: standardizing linguistic encoding
>
> Dear Eduard, [also addressing Philip and actually all... ]
>
> It's probably my conditioning as a member of various standardization
> bodies that makes red lights flash in my head upon reading that you are
> "developing a standard"... :-) I believe that standards are better
> "developed" (or, more precisely, codified) on the basis of existing best
> practices or other existing standards. As it is, I can observe that your
> proposed encoding mixes up the level of tokens with the level of word
> forms (in ISO MAF terminology[1]), and while it can be suitable for your
> purposes, it is far from optimal in standardization terms.
>
> [at this point, the camera pans out]
>
> This year promises to be quite exciting for the TEI Linguistics SIG[2],
> given that:
>
> (1) ISO LMF [3] is up for renewal and restructuring, and that several
> teams (among others, from ENeL, PARTHENOS, CLARIN, and LingSIG) are
> currently working on various modules for it,
>
> (2) ISO Tiger [4] is nearing publication (as in: weeks rather than
> months) and opening a way for ISO TEIger, a TEI serialization of the ISO
> model for syntactic encoding,
>
> (3) there is a rising push for streamlining inline linguistic markup,
> coming from, among others, Martin Mueller's Early Print Project, BBAW's
> existing practice (presented by Susanne Haaf at various TEI meetings),
> the Ancient Greek Dependency Treebank (represented in this mailing list
> by Giuseppe Celano, I believe), and now we learn of Philip Ströbel's
> project and yours. And there are others. A tiny reflex of that is
> contained at the LingSIG GitHub space [5], which is only meant as a
> _seed_ for collaborative effort rather than any personal statement.
>
> Andreas Witt and I are thinking of how to address and channel this
> boiling mass of initiatives. One possibility could be to target the
> upcoming TEI Members Meeting[6] and have a focused pre-conference
> workshop designed to formulate a very precise and very concrete proposal
> for grammatical encoding synchronized across inline, standoff and
> dictionary markup, a proposal that we could submit to the TEI Technical
> Council at the end of the day. "The day" seems distant, but if we want
> to have a serious proposal at the end of it, work should start about now.
>
> May I invite all interested parties to join the Linguistics SIG mailing
> list (by going to [7]) and GitHub space (by sending me, off-list, your
> github username), and to, well, have a go at it... :-)
>
> Best regards,
>
>    Piotr
>
> [1]:
> http://www.iso.org/iso/home/store/catalogue_tc/catalogue_
> detail.htm?csnumber=51934
> [2]: http://wiki.tei-c.org/index.php/SIG:TEI_for_Linguists
> [3]:
> http://www.iso.org/iso/home/store/catalogue_tc/catalogue_
> detail.htm?csnumber=68516
> [4]:
> http://www.iso.org/iso/home/store/catalogue_tc/catalogue_
> detail.htm?csnumber=62491
> [5]: https://github.com/LingSIG/wordAttributes/wiki
> [6]: http://members.tei-c.org/Events/meetings
>
>
>
>
>
>
> On 02/01/17 16:51, Eduard Drenth wrote:
> > Dear all,
> >
> > Here in Holland we are developing a standard to encode linguistic and
> > lemma information for various word situations using TEI. We have been
> > trying several solutions (tei:fs/tei:f, tei:interp, tei:span, ...) and
> > finaly chose for TEI customization which gives us standard xsd
> > validation, editor support and a simple focused solution. For linguistic
> > terminology we use as much as possible http://universaldependencies.org/
> .
> >
> >
> > We are curious as to what you think, see below for details. We hope this
> > solution may be of use for those who want to encode linguistic
> > information using TEI. Also this may help standardizing linguistic
> > encoding in TEI.
> >
> >
> > If this all is worthwhile I would like to donate/publish the solution
> > somewhere.
> >
> >
> > snippet customization:
> >
> >
> >             <schemaSpec ident="tdb" docLang="en" prefix="tei_"
> > xml:lang="en">
> >
> >                 ..
> >
> >                 ..
> >
> >                 <classSpec type="atts" ident="att.linguistics"
> > module="analytics">
> >
> >                     <attList>
> >                         <attDef ident="linguistics"
> > ns="http://www.fryske-akademy.org/grammar/1.0">
> >                             <desc>
> >                                 documentation....
> >                             </desc>
> >                             <datatype maxOccurs="unbounded">
> >                                 <dataRef key="teiata.enumerated"/>
> >                             </datatype>
> >                             <valList type="closed">
> >                                 <valItem ident="Features.Abbr">
> >                                     <desc>Boolean feature. Is this an
> > abbreviation?</desc>
> >                                 </valItem>
> >                                 <valItem ident="Features.Poss">
> >                                     <desc>Boolean feature of pronouns,
> > determiners or adjectives. It tells whether the word is
> possessive.</desc>
> >                                 </valItem>
> >                                 <valItem ident="PronType.Prs">
> >                                     <desc>personal pronoun or
> > determiner</desc>
> >                                 </valItem>
> >
> >                                  ..
> >
> >                                  ..
> >
> >
> > example word encoding:
> >
> >
> > <tei:w fa:linguistics="Pos.NOUN "
> > lemmaRef="inprogress://lemmasystem/Hollands/frik/1"
> > lemma="frik">Frik</tei:w>
> >
> >
> > example split word encoding:
> >
> >
> > <tei:w xml:id="staet-op-176" rendition="#split">staet</tei:w>
> >
> > <tei:w fa:linguistics="Pos.ADV "
> > lemmaRef="inprogress://lemmasystem/Hollands/al/3"
> > lemma="al">al</tei:w><tei:w>wringende</tei:w>
> >
> > <tei:w xml:id="staet-op-179" rendition="#split">op</tei:w>
> >
> > <tei:join result="w" scope="root" lemma="opstean" target="#staet-op-176
> > #staet-op-179" lemmaRef="inprogress://lemmasystem/Hollands/opstean/1"
> > fa:linguistics="th-si-pa Pos.VERB "/>
> >
> >
> > example word consist of more lemma's (we don't use this yet....):
> >
> >
> > <tei:choice>
> >   <tei:orig>
> >     <tei:w fa:linguistics=".....">aint</tei:w>
> >   </tei:orig>
> >   <tei:reg>
> >     <tei:w lemma="be" fa:linguistics="...">am</tei:w>
> >     <tei:w lemma="not" fa:linguistics="...">not</tei:w>
> >   </tei:reg>
> > </tei:choice>
> >
> >
> > Bye,
> >
> >
> > Eduard Drenth, Software Architekt
> >
> >
> > [email protected]
> >
> >
> > Doelestrjitte 8
> >
> > 8911 DX  Ljouwert
> >
> > +31 58 234 30 47
> >
> >
> > gpg: https://sks-keyservers.net/pks/lookup?op=get&search=
> 0x065EF82A1E02CC43
> >
>
> --
> Piotr Bański, Ph.D.
> Senior Researcher,
> Institut für Deutsche Sprache,
> R5 6-13
> 68-161 Mannheim, Germany
>