Re: collections, file names, and document order
Martin Honnen <[email protected]> Mon, 20 Dec 2021 00:21:31 +0100
| Newsgroups | gmane.text.xml.saxon.help |
|---|---|
| Message-ID | <[email protected]> |
This is a multi-part message in MIME format.
--===============0319489628168627258==
Content-Type: multipart/alternative;
boundary="------------eEZ0LWxaVFo3GCKMtCyt9CrG"
This is a multi-part message in MIME format.
--------------eEZ0LWxaVFo3GCKMtCyt9CrG
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: quoted-printable
Am 19.12.2021 um 23:46 schrieb Bauman, Syd:
> I have to admit, I am a little bit out of my depth, here, and am
> worried I am being an ingenious fool. So some affirmation or
> correction seems in order. I am using Saxon-HE 10.3J on the command line=
.
>
> I would like to process a subset of files in a set of directories.
> Seems like a job for the collection() function. But the process I am
> thinking of (as most of my processes) needs to know the input file=E2=80=
=99s
> path. Well, the uri-collection() function seems like the ticket, eh?
>
> But I note the documentation
> <https://www.saxonica.com/html/documentation10/sourcedocs/collections/>
> says that the nodes returned by collection() (and I presume
> uri-collection(), too?) are not necessarily in document order. First
> off, I do not entirely grok what =E2=80=9Cdocument order=E2=80=9D means =
with respect
> to a collection of files. (Perhaps =E2=80=9Cthe order your OS returns th=
em in=E2=80=9D
> or =E2=80=9Cthe same order as you got last time=E2=80=9D or =E2=80=9Calp=
habetic order=E2=80=9D?) But
> it doesn=E2=80=99t really matter to me what order the files are presente=
d in,
> but to use uri-collection() to get the names and collection() to get
> the corresponding file=E2=80=99s contents, it matters that the two funct=
ions
> return the file(name)s in the /same/ order, whatever it is.
>
> The attached (useless) program demonstrates what I am trying to do. In
> this case, process all of the files in my ~/Documents/whatever/one/,
> ~/Documents/whatever/two/, and ~/Documents/whatever/three/ directories
> that start with the letter =E2=80=98r=E2=80=99, have a dot in the middle=
, and end with
> the extension =E2=80=9C.odd=E2=80=9D, =E2=80=9C.tei=E2=80=9D, or =E2=80=
=9C.xml=E2=80=9D. Note to TEI folks =E2=80=94 for the
> purposes of this discussion /every/ document has an outermost element
> of <TEI>, and /each/ header has one and only one <idno> in the
> <publicationStmt>. Yes, TEI allows variations, but this is about
> collection(), not how to find stuff in a TEI document. =F0=9F=99=82
>
> Questions, in rough order of importance (in some sense of =E2=80=9Cimpor=
tance=E2=80=9D
> :-) =E2=80=94
>
> * On line 21 I am dutifully grabbing the outermost TEI elements
> using |!*|=E2=80=8B instead of |/*|=E2=80=8B, in order not to force =
Saxon to hold
> the entire collection in memory at once. But is that a problem? Do
> I /want/ to force them to be in order?
> * This program rests entirely on the idea that collection() and
> uri-collection() return stuff in the same order, whatever that
> order is. Is that true in general?
> * If it is true in general, Is it rendered false by
> =E2=80=9Con-error=3Dignore=E2=80=9D? If we had a file whose URI uri-=
collection()
> could grab, but whose content caused an error for collection(),
> would the two sequences have a different number of items? (In
> which case my comparison by position would fail.)
> * Is using position() on line 50 OK, or should I really be using
> index-of( $input_TEIs, . )?
> * Is there some completely better way to do this, like iterating
> over $input_URIs and reading each one in with doc(), and handing
> it to a template to decorate its outermost element with
> =E2=80=9Cfile=3D{$file}=E2=80=9D?
> * If I declare $idx on line 50 as xs:positiveInteger I get yelled
> at, because it needs to be an xs:integer. What=E2=80=99s the point o=
f
> having a datatype system with inheritance if I can=E2=80=99t use a d=
erived
> datatype to be more precise?
> * Along the same lines, I note that $myURI (line 51) is declared as
> xs:anyURI, but is not cast to a string before being handed to
> tokenize(), the signature for which says it wants an xs:string.
>
> Hope I=E2=80=99m not being too stupid. Thoughts appreciated.
If I had a sequence of URIs and wanted to load and process the documents
the URIs refer to in the order of the URIs in the sequence I would
certainly use
=C2=A0 $input_URIs ! doc(.)
to ensure that.
The URI can always be read out during processing with the document-uri()
function (I think you need to feed it the root node if it is not the
context node).
I would also say that the=C2=A0 !* is the right way to keep processing the
root elements in the order of the input documents, as you want that you
need to use ! (or for .. return) and not /, that makes the order
arbitrary as it uses document order which doesn't exist between elements
from different documents.
To avoid loading the entire collection I think PE and EE have
discard-document, but HE doesn't support that. Though XSLT 3 has
xsl:source-document also I think, if I remember that right (or does that
only apply to streaming?) it doesn't cache documents like `doc` does.
I don't know whether collection ensures the same order as uri-collection.
--------------eEZ0LWxaVFo3GCKMtCyt9CrG
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<html>
<head>
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3DUTF-=
8">
</head>
<body>
<p><br>
</p>
<div class=3D"moz-cite-prefix">Am 19.12.2021 um 23:46 schrieb Bauman,
Syd:<br>
</div>
<blockquote type=3D"cite"
cite=3D"mid:MN2PR06MB6094562E7E50A567E89C3971FC7A9-AF3PXZdH1KB0WTTyG1/[email protected]=
prod.outlook.com">
<meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3DUT=
F-8">
<style type=3D"text/css" style=3D"display:none;">P {margin-top:0;mar=
gin-bottom:0;}</style>
<div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
I have to admit, I am a little bit out of my depth, here, and am
worried I am being an ingenious fool. So some affirmation or
correction seems in order. I am using Saxon-HE 10.3J on the
command line.</div>
<div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
I would like to process a subset of files in a set of
directories. Seems like a job for the collection() function. But
the process I am thinking of (as most of my processes) needs to
know the input file=E2=80=99s path. Well, the uri-collection() fun=
ction
seems like the ticket, eh?</div>
<div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
But I note the <a
href=3D"https://www.saxonica.com/html/documentation10/sourcedocs/collectio=
ns/"
title=3D"https://www.saxonica.com/html/documentation10/sourcedocs/collecti=
ons/"
moz-do-not-send=3D"true">
documentation</a> says that the nodes returned by collection()
(and I presume uri-collection(), too?) are not necessarily in
document order. First off, I do not entirely grok what =E2=80=9Cdo=
cument
order=E2=80=9D means with respect to a collection of files. (Perha=
ps
=E2=80=9Cthe order your OS returns them in=E2=80=9D or =E2=80=9Cth=
e same order as you
got last time=E2=80=9D or =E2=80=9Calphabetic order=E2=80=9D?) But=
it doesn=E2=80=99t really
matter to me what order the files are presented in, but to use
uri-collection() to get the names and collection() to get the
corresponding file=E2=80=99s contents, it matters that the two fun=
ctions
return the file(name)s in the <i>same</i> order, whatever it
is.</div>
<div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
The attached (useless) program demonstrates what I am trying to
do. In this case, process all of the files in my
~/Documents/whatever/one/, ~/Documents/whatever/two/, and
~/Documents/whatever/three/ directories that start with the
letter =E2=80=98r=E2=80=99, have a dot in the middle, and end with=
the extension
=E2=80=9C.odd=E2=80=9D, =E2=80=9C.tei=E2=80=9D, or =E2=80=9C.xml=
=E2=80=9D. Note to TEI folks =E2=80=94 for the purposes
of this discussion
<i>every</i> document has an outermost element of <TEI>,
and <i>each</i> header has one and only one <idno> in the
<publicationStmt>. Yes, TEI allows variations, but this is
about collection(), not how to find stuff in a TEI document.
<span id=3D"=F0=9F=99=82" contenteditable=3D"false">=F0=9F=99=82</=
span><br>
</div>
<div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
Questions, in rough order of importance (in some sense of
=E2=80=9Cimportance=E2=80=9D :-) =E2=80=94</div>
<div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0);">
<ul>
<li>On line 21 I am dutifully grabbing the outermost TEI
elements using <code>!*</code>=E2=80=8B instead of
<code>/*</code>=E2=80=8B, in order not to force Saxon to hold =
the
entire collection in memory at once. But is that a problem?
Do I
<i>want</i> to force them to be in order?</li>
<li>This program rests entirely on the idea that collection()
and uri-collection() return stuff in the same order,
whatever that order is. Is that true in general?</li>
<li>If it is true in general, Is it rendered false by
=E2=80=9Con-error=3Dignore=E2=80=9D? If we had a file whose UR=
I
uri-collection() could grab, but whose content caused an
error for collection(), would the two sequences have a
different number of items? (In which case my comparison by
position would fail.)</li>
<li>Is using position() on line 50 OK, or should I really be
using=C2=A0<span style=3D"font-family: "Courier New"=
,
monospace;">index-of( $input_TEIs, . )</span>?</li>
<li>Is there some completely better way to do this, like
iterating over $input_URIs and reading each one in with
doc(), and handing it to a template to decorate its
outermost element with =E2=80=9Cfile=3D{$file}=E2=80=9D?<br>
</li>
<li>If I declare $idx on line 50 as xs:positiveInteger I get
yelled at, because it needs to be an xs:integer. What=E2=80=99=
s the
point of having a datatype system with inheritance if I
can=E2=80=99t use a derived datatype to be more precise?</li>
<li>Along the same lines, I note that $myURI (line 51) is
declared as xs:anyURI, but is not cast to a string before
being handed to tokenize(), the signature for which says it
wants an xs:string.</li>
</ul>
<div>Hope I=E2=80=99m not being too stupid. Thoughts appreciated.<=
/div>
</div>
</blockquote>
<p><br>
</p>
<p>If I had a sequence of URIs and wanted to load and process the
documents the URIs refer to in the order of the URIs in the
sequence I would certainly use <br>
</p>
<p>=C2=A0 $input_URIs ! doc(.)</p>
<p>to ensure that.</p>
<p>The URI can always be read out during processing with the
document-uri() function (I think you need to feed it the root node
if it is not the context node).</p>
<p><br>
</p>
<p>I would also say that the=C2=A0 !* is the right way to keep process=
ing
the root elements in the order of the input documents, as you want
that you need to use ! (or for .. return) and not /, that makes
the order arbitrary as it uses document order which doesn't exist
between elements from different documents.</p>
<p> <br>
</p>
<p>To avoid loading the entire collection I think PE and EE have
discard-document, but HE doesn't support that. Though XSLT 3 has
xsl:source-document also I think, if I remember that right (or
does that only apply to streaming?) it doesn't cache documents
like `doc` does.</p>
<p><br>
</p>
<p>I don't know whether collection ensures the same order as
uri-collection.<br>
</p>
</body>
</html>
--------------eEZ0LWxaVFo3GCKMtCyt9CrG--
--===============0319489628168627258==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
--===============0319489628168627258==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
saxon-help mailing list archived at http://saxon.markmail.org/
[email protected]
https://lists.sourceforge.net/lists/listinfo/saxon-help
--===============0319489628168627258==--