Namespaces are dropped in XMP stream. XML invalid

Jörn Willhöft <[email protected]> Tue, 5 Nov 2024 12:25:40 +0100 (CET)
Newsgroups gmane.text.xml.fop.user
Message-ID <[email protected]>
------=_Part_162676_2140693864.1730805940191
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Dear list members,
=20
I need to create PDF/A-3B compliant documents with special references to an=
 embedded document (ZUGFeRD compliant invoices). Currently, this fails in V=
eraPDF with some very vague and rather misleading validation errors:
* "Specification: ISO 19005-3:2012, Clause: 6.6.2.1, Test number: 5
All metadata streams present in the PDF shall conform to the XMP Specificat=
ion. The XMP package must be encoded as UTF-8"
* "Specification: ISO 19005-3:2012, Clause: 6.6.4, Test number: 1
The PDF/A version and conformance level of a file shall be specified using =
the PDF/A Identification extension schema"
* "Specification: ISO 19005-3:2012, Clause: 6.6.2.1, Test number: 4
All metadata streams present in the PDF shall conform to the XMP Specificat=
ion. All content of all XMP packets shall be well-formed, as defined by Ext=
ensible Markup Language (XML) 1.0 (Third Edition), 2.1, and the RDF/XML Syn=
tax Specification (Revised)"
=20
I finally tracked this down to FOP, which dropped some necessary namespaces=
 in the created XMP stream. This can be reproduced with the corresponding e=
xample from the FOP homepage (https://xmlgraphics.apache.org/fop/2.10/metad=
ata.html). E.g. here is the fo:

<fo:simple-page-master master-name=3D"simple">
  <fo:region-body/>
  <pdf:page page-numbers=3D"*">
    <x:xmpmeta xmlns:x=3D"adobe:ns:meta/">
      <rdf:RDF xmlns:rdf=3D"http://www.w3.org/1999/02/22-rdf-syntax-ns#" xm=
lns:abc=3D"http://www.abc.de/abc/">
        <rdf:Description rdf:about=3D"" abc:def=3D"val"/>
        <rdf:Description rdf:about=3D"" xmlns:pdfaExtension=3D"http://www.a=
iim.org/pdfa/ns/extension/"
                         xmlns:pdfaSchema=3D"http://www.aiim.org/pdfa/ns/sc=
hema#"
                         xmlns:pdfaProperty=3D"http://www.aiim.org/pdfa/ns/=
property#">
          <pdfaExtension:schemas>
            <rdf:Bag>
              <rdf:li rdf:parseType=3D"Resource">
                <pdfaSchema:property>
                  <rdf:Seq>
                    <rdf:li rdf:parseType=3D"Resource">
                      <pdfaProperty:name>split</pdfaProperty:name>
                    </rdf:li>
                  </rdf:Seq>
                </pdfaSchema:property>
              </rdf:li>
            </rdf:Bag>
          </pdfaExtension:schemas>
        </rdf:Description>
      </rdf:RDF>
    </x:xmpmeta>
  </pdf:page>
</fo:simple-page-master>

And this is returned when I run pdfinfo -meta example.pdf:
<?xpacket begin=3D"" id=3D"W5M0MpCehiHzreSzNTczkc9d"?><x:xmpmeta xmlns:x=3D=
"adobe:ns:meta/">
  <rdf:RDF xmlns:rdf=3D"http://www.w3.org/1999/02/22-rdf-syntax-ns#">
     <rdf:RDF xmlns:abc=3D"http://www.abc.de/abc/" abc:def=3D"val" rdf:abou=
t=3D""/>
     <rdf:RDF xmlns:dc=3D"http://purl.org/dc/elements/1.1/" rdf:about=3D"">
        <dc:format>application/pdf</dc:format>
        <dc:language>
           <rdf:Bag>
              <rdf:li>x-unknown</rdf:li>
           </rdf:Bag>
        </dc:language>
        <dc:date>
           <rdf:Seq>
              <rdf:li>2024-11-05T11:46:27+01:00</rdf:li>
           </rdf:Seq>
        </dc:date>
     </rdf:RDF>
     <rdf:RDF xmlns:pdf=3D"http://ns.adobe.com/pdf/1.3/" rdf:about=3D"">
        <pdf:Producer>Apache FOP Version 2.10</pdf:Producer>
        <pdf:PDFVersion>1.4</pdf:PDFVersion>
     </rdf:RDF>
     <rdf:RDF xmlns:pdfaExtension=3D"http://www.aiim.org/pdfa/ns/extension/=
" rdf:about=3D"">
        <pdfaExtension:schemas>
           <rdf:Bag>
              <rdf:li rdf:parseType=3D"Resource">
                 <pdfaSchema:property>
                    <rdf:Seq>
                       <rdf:li rdf:parseType=3D"Resource">
                          <pdfaProperty:name>split</pdfaProperty:name>
                       </rdf:li>
                    </rdf:Seq>
                 </pdfaSchema:property>
              </rdf:li>
           </rdf:Bag>
        </pdfaExtension:schemas>
     </rdf:RDF>
     <rdf:RDF xmlns:pdfaid=3D"http://www.aiim.org/pdfa/ns/id/" rdf:about=3D=
"">
        <pdfaid:conformance>B</pdfaid:conformance>
        <pdfaid:part>3</pdfaid:part>
     </rdf:RDF>
     <rdf:RDF xmlns:xmp=3D"http://ns.adobe.com/xap/1.0/" rdf:about=3D"">
        <xmp:MetadataDate>2024-11-05T11:46:27+01:00</xmp:MetadataDate>
        <xmp:CreateDate>2024-11-05T11:46:27+01:00</xmp:CreateDate>
     </rdf:RDF>
  </rdf:RDF>
</x:xmpmeta>
<?xpacket end=3D"r"?>

As you can see, the two namespaces xmlns:pdfaSchema=3D"http://www.aiim.org/=
pdfa/ns/schema#"   and xmlns:pdfaProperty=3D"http://www.aiim.org/pdfa/ns/pr=
operty#"
are dropped from the metadata, while the prefixes are still used. Thus, the=
 XML is invalid and so is the PDF/A.
This is pretty unfortunate, as I don't have any workaround for this. I am u=
sing Apache FOP 2.10 in the context of Apache Camel 4.8.1. Any help would b=
e greatly appreciated.
=20

With kind regards,

   J=C3=B6rn Willh=C3=B6ft
------=_Part_162676_2140693864.1730805940191
MIME-Version: 1.0
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<!doctype html>
<html>
 <head>=20
  <meta charset=3D"UTF-8">=20
 </head>
 <body>
  <div class=3D"default-style">
   Dear list members,
  </div>=20
  <div class=3D"default-style">
   &nbsp;
  </div>=20
  <div class=3D"default-style">
   I need to create PDF/A-3B compliant documents with special references to=
 an embedded document (ZUGFeRD compliant invoices). Currently, this fails i=
n VeraPDF with some very vague and rather misleading validation errors:
  </div>=20
  <div class=3D"default-style">=20
   <div class=3D"default-style">
    * "Specification: ISO 19005-3:2012, Clause: 6.6.2.1, Test number: 5=20
    <br>All metadata streams present in the PDF shall conform to the XMP Sp=
ecification. The XMP package must be encoded as UTF-8"
   </div>=20
   <div class=3D"default-style">
    * "Specification: ISO 19005-3:2012, Clause: 6.6.4, Test number: 1=20
    <br>The PDF/A version and conformance level of a file shall be specifie=
d using the PDF/A Identification extension schema"
   </div>=20
   <div class=3D"default-style">
    * "Specification: ISO 19005-3:2012, Clause: 6.6.2.1, Test number: 4=20
    <br>All metadata streams present in the PDF shall conform to the XMP Sp=
ecification. All content of all XMP packets shall be well-formed, as define=
d by Extensible Markup Language (XML) 1.0 (Third Edition), 2.1, and the RDF=
/XML Syntax Specification (Revised)"
   </div>=20
   <div class=3D"default-style">
    &nbsp;
   </div>=20
   <div class=3D"default-style">
    I finally tracked this down to FOP, which dropped some necessary namesp=
aces in the created XMP stream. This can be reproduced with the correspondi=
ng example from the FOP homepage (<a href=3D"https://xmlgraphics.apache.org=
/fop/2.10/metadata.html">https://xmlgraphics.apache.org/fop/2.10/metadata.h=
tml</a>). E.g. here is the fo:
   </div>=20
   <div class=3D"default-style">=20
    <pre><em>&lt;fo:simple-page-master master-name=3D"simple"&gt;
  &lt;fo:region-body/&gt;
  &lt;pdf:page page-numbers=3D"*"&gt;
    &lt;x:xmpmeta xmlns:x=3D"adobe:ns:meta/"&gt;
      &lt;rdf:RDF xmlns:rdf=3D"http://www.w3.org/1999/02/22-rdf-syntax-ns#"=
 xmlns:abc=3D"http://www.abc.de/abc/"&gt;
        &lt;rdf:Description rdf:about=3D"" abc:def=3D"val"/&gt;
        &lt;rdf:Description rdf:about=3D"" xmlns:pdfaExtension=3D"http://ww=
w.aiim.org/pdfa/ns/extension/"
                         xmlns:pdfaSchema=3D"http://www.aiim.org/pdfa/ns/sc=
hema#"
                         xmlns:pdfaProperty=3D"http://www.aiim.org/pdfa/ns/=
property#"&gt;
          &lt;pdfaExtension:schemas&gt;
            &lt;rdf:Bag&gt;
              &lt;rdf:li rdf:parseType=3D"Resource"&gt;
                &lt;pdfaSchema:property&gt;
                  &lt;rdf:Seq&gt;
                    &lt;rdf:li rdf:parseType=3D"Resource"&gt;
                      &lt;pdfaProperty:name&gt;split&lt;/pdfaProperty:name&=
gt;
                    &lt;/rdf:li&gt;
                  &lt;/rdf:Seq&gt;
                &lt;/pdfaSchema:property&gt;
              &lt;/rdf:li&gt;
            &lt;/rdf:Bag&gt;
          &lt;/pdfaExtension:schemas&gt;
        &lt;/rdf:Description&gt;
      &lt;/rdf:RDF&gt;
    &lt;/x:xmpmeta&gt;
  &lt;/pdf:page&gt;
&lt;/fo:simple-page-master&gt;
</em></pre>=20
   </div>=20
   <div class=3D"default-style">
    And this is returned when I run pdfinfo -meta example.pdf:
   </div>=20
   <div class=3D"default-style">
    <span style=3D"font-family: monospace;"><span style=3D"color: #000000; =
background-color: #ffffff;" data-darkreader-inline-color=3D"" data-darkread=
er-inline-bgcolor=3D"">&lt;?xpacket begin=3D</span><span style=3D"color: #0=
00000; background-color: #ffffff;" data-darkreader-inline-color=3D"" data-d=
arkreader-inline-bgcolor=3D"">"</span><span style=3D"color: #000000; backgr=
ound-color: #ffffff;" data-darkreader-inline-color=3D"" data-darkreader-inl=
ine-bgcolor=3D"">" id=3D"W5M0MpCehiHzreSzNTczkc9d"?&gt;&lt;x:xmpmeta xmlns:=
x=3D"adobe:ns:meta/"&gt; </span><br>&nbsp;&nbsp;&lt;rdf:RDF xmlns:rdf=3D"ht=
tp://www.w3.org/1999/02/22-rdf-syntax-ns#"&gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;=
&nbsp;&lt;rdf:RDF xmlns:abc=3D"http://www.abc.de/abc/" abc:def=3D"val" rdf:=
about=3D""/&gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;rdf:RDF xmlns:dc=3D"h=
ttp://purl.org/dc/elements/1.1/" rdf:about=3D""&gt; <br>&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;dc:format&gt;application/pdf&lt;/dc:format=
&gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;dc:language&gt=
; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt=
;rdf:Bag&gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;&lt;rdf:li&gt;x-unknown&lt;/rdf:li&gt; <br>&nbsp;=
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/rdf:Bag&gt=
; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/dc:language&gt; =
<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;dc:date&gt; <br>&nb=
sp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;rdf:Seq&=
gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&lt;rdf:li&gt;2024-11-05T11:46:27+01:00&lt;/rdf:li&gt; <br=
>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/rdf=
:Seq&gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/dc:date&g=
t; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/rdf:RDF&gt; <br>&nbsp;&nbsp;&nbsp=
;&nbsp;&nbsp;&lt;rdf:RDF xmlns:pdf=3D"http://ns.adobe.com/pdf/1.3/" rdf:abo=
ut=3D""&gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;pdf:Pro=
ducer&gt;Apache FOP Version 2.10&lt;/pdf:Producer&gt; <br>&nbsp;&nbsp;&nbsp=
;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;pdf:PDFVersion&gt;1.4&lt;/pdf:PDFVersion=
&gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/rdf:RDF&gt; <br>&nbsp;&nbsp;&nb=
sp;&nbsp;&nbsp;&lt;rdf:RDF xmlns:pdfaExtension=3D"http://www.aiim.org/pdfa/=
ns/extension/" rdf:about=3D""&gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&lt;pdfaExtension:schemas&gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;=
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;rdf:Bag&gt; <br>&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;rdf:li=
 rdf:parseType=3D"Resource"&gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nb=
sp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;pdfaSche=
ma:property&gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;rdf:Se=
q&gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp=
;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&l=
t;rdf:li rdf:parseType=3D"Resource"&gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;pdfaProperty:name&gt;=
split&lt;/pdfaProperty:name&gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nb=
sp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;=
&nbsp;&nbsp;&nbsp;&nbsp;&lt;/rdf:li&gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&lt;/rdf:Seq&gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/pdfaSche=
ma:property&gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/rdf:li&gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;&n=
bsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/rdf:Bag&gt; <br>&nbsp;&nbsp;&n=
bsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/pdfaExtension:schemas&gt; <br>&nbsp;=
&nbsp;&nbsp;&nbsp;&nbsp;&lt;/rdf:RDF&gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;=
&lt;rdf:RDF xmlns:pdfaid=3D"http://www.aiim.org/pdfa/ns/id/" rdf:about=3D""=
&gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;pdfaid:conform=
ance&gt;B&lt;/pdfaid:conformance&gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&lt;pdfaid:part&gt;3&lt;/pdfaid:part&gt; <br>&nbsp;&nbsp;&nbs=
p;&nbsp;&nbsp;&lt;/rdf:RDF&gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;rdf:RD=
F xmlns:xmp=3D"http://ns.adobe.com/xap/1.0/" rdf:about=3D""&gt; <br>&nbsp;&=
nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;xmp:MetadataDate&gt;2024-11-05=
T11:46:27+01:00&lt;/xmp:MetadataDate&gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;=
&nbsp;&nbsp;&nbsp;&lt;xmp:CreateDate&gt;2024-11-05T11:46:27+01:00&lt;/xmp:C=
reateDate&gt; <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&lt;/rdf:RDF&gt; <br>&nbsp;=
&nbsp;&lt;/rdf:RDF&gt; <br>&lt;/x:xmpmeta&gt; <br>&lt;?xpacket end=3D"r"?&g=
t;<br><br></span>
   </div>=20
   <div class=3D"default-style">
    As you can see, the two namespaces <em>xmlns:pdfaSchema=3D"http://www.a=
iim.org/pdfa/ns/schema#"&nbsp;&nbsp; and xmlns:pdfaProperty=3D"http://www.a=
iim.org/pdfa/ns/property#"</em>
   </div>=20
   <div class=3D"default-style">
    are dropped from the metadata, while the prefixes are still used. Thus,=
 the XML is invalid and so is the PDF/A.
   </div>=20
   <div class=3D"default-style">
    This is pretty unfortunate, as I don't have any workaround for this. I =
am using Apache FOP 2.10 in the context of Apache Camel 4.8.1. Any help wou=
ld be greatly appreciated.
   </div>=20
   <div class=3D"default-style">
    &nbsp;
   </div>=20
  </div>=20
  <div class=3D"io-ox-signature">=20
   <p>With kind regards,</p>=20
   <p>&nbsp;&nbsp; J=C3=B6rn Willh=C3=B6ft</p>=20
  </div>
 </body>
</html>
------=_Part_162676_2140693864.1730805940191--