Re: Index field - Computed field function access to the same collection?

--===============4333692617503777739==
Content-Type: multipart/alternative; boundary="0000000000007addaa064274fce3"

--0000000000007addaa064274fce3
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Dear Lars,

There are two possible issues at work.

The first is subtle and I only discovered it over time: computed fields in
eXist-db use the Lucene index, which in turn requires a text() node to be
available on the element containing an attribute you may wish to index. A
sample example demonstrates this:

<item corresp=3D"foo"/>  -> cannot compute a field from the value in @corre=
sp
<item corresp=3D"foo">bar</item> -> can compute a field from the value
in @corresp

This may or may not affect your indexing situation.

Compounded with this is a second issue that you indicate. An index being
performed on a given document (say "X") cannot access the rest of the
contents held in the same immediate collection that contains "X". Thus
computed fields fail to populate when referring to documents in the same
immediate collection. My own work-around is to create separate collections
for documents which may be needed for indexing. For example, if I have a
corpus of TEI letters in collection /data/corpus, I will keep separate xml
files containing, for example, my ListPerson or List Place in
/data/listPerson and /data/listPlace respectively. This makes them fully
available for use in computed fields. They cannot be collections within
collections. Note as well, that if one triggers an index on all of /data/
with a collection.xconf in /data/ one would face the same problem.
Therefore each collection needs its own collection.xconf.

If your situation is like mine the first time, it means rebuilding the
collections, and refactoring and rewiring some code (and now I keep all my
XQuery paths in "global-like" variables so I only touch them in one place
if a change is needed.)

I hope this helps.

Best,

JPR



On Fri, Oct 31, 2025 at 12:21=E2=80=AFPM Lars Scheideler <scheideler@saw-le=
ipzig.de>
wrote:

> Hello,
>
> I want to index a collection and need a field that contains specific
> computed values from the same collection.
> I have various TEI XML files with different biblStruct types. Some of
> these files refer to other biblStructs via ref elements. I need the IDs o=
f
> the biblStructs that point to the current, indexed biblStruct.
> However, when I reindex the collection, the fields almost exclusively
> contain =E2=80=9Cempty=E2=80=9D values.
>
> Here the code:
>
> <collection xmlns=3D"http://exist-db.org/collection-config/1.0"
> <http://exist-db.org/collection-config/1.0>>
>     <index xmlns:tei=3D"http://www.tei-c.org/ns/1.0"
> <http://www.tei-c.org/ns/1.0> xmlns:xs=3D"http://www.w3.org/2001/XMLSchem=
a"
> <http://www.w3.org/2001/XMLSchema>>
>         <lucene>
>             <module uri=3D"http://application.de/xquery/facet-utils"
> <http://application.de/xquery/facet-utils> prefix=3D"fu" at=3D
> "xmldb:exist:///db/apps/application/modules/index-facet-utils.xqm"/>
>             <text qname=3D"tei:biblStruct">
>                 <field name=3D"workgroupid" expression=3D
> "fu:get-workgroup-id(.)"/>
>             </text>
>         </lucene>
>     </index>
> </collection>
>
>
> declare function fu:get-workgroup-id($biblStruct as element()) as
> xs:string* {
>     let $corresp as xs:string :=3D $biblStruct/@corresp/string()
>     let $workgroupsIds as xs:string* :=3D
> collection("/db/projects/application/data/bibls")//tei:biblStruct[@type
> eq 'dataset'][.//tei:ref[@type eq 'relDocRef']/@target/string() =3D
> $corresp]/@xml:id/string()
>     let $workgroupsIdsJoined :=3D string-join($workgroupsIds,';')
>     return
>         if($workgroupsIdsJoined ne "") then $workgroupsIdsJoined else
> "empty"
> };
>
> I assume that I do not have access to the same collection that is actuall=
y
> indexed or to specific attributes/elements (this case is not
> described/addressed/mentioned in either the documentation or the book), b=
ut
> the function takes far too long for live productive use.
>
> Thanks for the help.
>
> Best regards
> _______________________________________________
> Exist-open mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/exist-open
>

--0000000000007addaa064274fce3
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Dear Lars,</div><div><br></div><div>There are two pos=
sible=C2=A0issues at work.</div><div><br></div><div>The first is subtle and=
 I only discovered=C2=A0it over time: computed fields in eXist-db use the L=
ucene index, which in turn requires a text() node to be available=C2=A0on t=
he element containing an attribute you may wish to index. A sample=C2=A0exa=
mple demonstrates this:</div><div><br></div><div>&lt;item corresp=3D&quot;f=
oo&quot;/&gt;=C2=A0 -&gt; cannot compute a field from the value in=C2=A0@co=
rresp</div><div>&lt;item corresp=3D&quot;foo&quot;&gt;bar&lt;/item&gt; -&gt=
; can compute a field from the value in=C2=A0@corresp</div><div><br></div><=
div>This may or may not affect your indexing situation.</div><div><br></div=
><div>Compounded with this is a second issue that you indicate. An index be=
ing performed on a given document (say &quot;X&quot;) cannot access the res=
t of the contents held in the same=C2=A0immediate collection that=C2=A0cont=
ains &quot;X&quot;. Thus computed fields fail to populate when referring to=
 documents in the same immediate collection. My own work-around is to creat=
e separate collections for documents which may be needed for indexing. For =
example, if I have a corpus of TEI letters in collection /data/corpus, I wi=
ll keep separate xml files containing, for example, my ListPerson or List P=
lace in /data/listPerson and /data/listPlace respectively. This makes them =
fully available for use in computed fields. They cannot be collections with=
in collections. Note as well, that if one triggers an index on all of /data=
/ with a collection.xconf in /data/ one would face the same problem. Theref=
ore each collection needs its own=C2=A0collection.xconf.</div><div><br></di=
v>If your situation is like mine the first time, it means rebuilding the co=
llections, and refactoring and rewiring some code (and now I keep all my XQ=
uery paths in &quot;global-like&quot; variables so I only touch them in one=
 place if a change is needed.)=C2=A0<div><br></div><div>I hope this helps.<=
/div><div><br></div><div>Best,</div><div><br></div><div>JPR</div><div><br><=
/div><div><br></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" c=
lass=3D"gmail_attr">On Fri, Oct 31, 2025 at 12:21=E2=80=AFPM Lars Scheidele=
r &lt;<a href=3D"mailto:[email protected]" target=3D"_blank">scheid=
[email protected]</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quot=
e" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204)=
;padding-left:1ex"><u></u>

 =20

   =20
 =20
  <div style=3D"padding-bottom:1px">
    <p><font face=3D"Source Sans 3">Hello,</font></p>
    <p><font face=3D"Source Sans 3">I want to index a collection and need
        a field that contains specific computed values from the same
        collection.=C2=A0<br>
        I have various TEI XML files with different biblStruct types.
        Some of these files refer to other biblStructs via ref elements.
        I need the IDs of the biblStructs that point to the current,
        indexed biblStruct.<br>
        However, when I reindex the collection, the fields almost
        exclusively contain =E2=80=9Cempty=E2=80=9D values.=C2=A0</font></p=
>
    <p><font face=3D"Source Sans 3">Here the code:=C2=A0</font></p>
    <p><font face=3D"Source Sans 3">&lt;collection
        xmlns=3D<a href=3D"http://exist-db.org/collection-config/1.0" targe=
t=3D"_blank">&quot;http://exist-db.org/collection-config/1.0&quot;</a>&gt;<=
br>
        =C2=A0 =C2=A0 &lt;index <a>xmlns:tei=3D</a><a href=3D"http://www.te=
i-c.org/ns/1.0" target=3D"_blank">&quot;http://www.tei-c.org/ns/1.0&quot;</=
a>
        xmlns:xs=3D<a href=3D"http://www.w3.org/2001/XMLSchema" target=3D"_=
blank">&quot;http://www.w3.org/2001/XMLSchema&quot;</a>&gt;<br>
        =C2=A0 =C2=A0 =C2=A0 =C2=A0 &lt;lucene&gt;<br>
        =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 &lt;module
        uri=3D<a href=3D"http://application.de/xquery/facet-utils" target=
=3D"_blank">&quot;http://application.de/xquery/facet-utils&quot;</a> prefix=
=3D&quot;fu&quot;
at=3D<a>&quot;xmldb:exist:///db/apps/application/modules/index-facet-utils.=
xqm&quot;</a>/&gt;<br>
        =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 &lt;text qname=3D<a>&quot=
;tei:biblStruct&quot;</a>&gt;<br>
        =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 &lt;field n=
ame=3D&quot;workgroupid&quot;
        expression=3D<a>&quot;fu:get-workgroup-id(.)&quot;</a>/&gt;<br>
        =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 &lt;/text&gt;<br>
        =C2=A0 =C2=A0 =C2=A0 =C2=A0 &lt;/lucene&gt;<br>
        =C2=A0 =C2=A0 &lt;/index&gt;<br>
        &lt;/collection&gt;<br>
        <br>
        <br>
        declare function <a>fu:get-workgroup-id($biblStruct</a> as element(=
))
        as <a>xs:string*</a> {<br>
        =C2=A0 =C2=A0 let $corresp as <a>xs:string</a> :=3D $biblStruct/@co=
rresp/string()<br>
        =C2=A0 =C2=A0 let $workgroupsIds as <a>xs:string*</a> :=3D
        collection(&quot;/db/projects/application/data/bibls&quot;)//<a>tei=
:biblStruct</a>[@type
        eq &#39;dataset&#39;][.//<a>tei:ref</a>[@type eq &#39;relDocRef&#39=
;]/@target/string()
        =3D $corresp]/@<a>xml:id/string()</a><br>
        =C2=A0 =C2=A0 let $workgroupsIdsJoined :=3D string-join($workgroups=
Ids,&#39;;&#39;)<br>
        =C2=A0 =C2=A0 return<br>
        =C2=A0 =C2=A0 =C2=A0 =C2=A0 if($workgroupsIdsJoined ne &quot;&quot;=
) then $workgroupsIdsJoined
        else &quot;empty&quot;<br>
        };</font></p>
    <p><font face=3D"Source Sans 3">I assume that I do not have access to
        the same collection that is actually indexed or to specific
        attributes/elements (this case is not
        described/addressed/mentioned in either the documentation or the
        book), but the function takes far too long for live productive
        use.=C2=A0</font></p>
    <p><font face=3D"Source Sans 3">Thanks for the help.</font></p>
    <p><font face=3D"Source Sans 3">Best regards=C2=A0</font></p>
  </div>
  <u></u><u></u>

_______________________________________________<br>
Exist-open mailing list<br>
<a href=3D"mailto:[email protected]" target=3D"_blank">Exist=
[email protected]</a><br>
<a href=3D"https://lists.sourceforge.net/lists/listinfo/exist-open" rel=3D"=
noreferrer" target=3D"_blank">https://lists.sourceforge.net/lists/listinfo/=
exist-open</a><br>
</blockquote></div>

--0000000000007addaa064274fce3--


--===============4333692617503777739==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline


--===============4333692617503777739==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Exist-open mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/exist-open

--===============4333692617503777739==--