RE: RE: Fop batch generate report performance
"Simon Steiner" <[email protected]> Tue, 4 Jun 2024 15:19:17 +0100
| Newsgroups | gmane.text.xml.fop.user |
|---|---|
| Message-ID | <[email protected]> |
------=_NextPart_000_0014_01DAB692.91914C70
Content-Type: text/plain;
charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Hi,
=20
What about:
FopFactory fopFactory =3D createFopFactory();
requestData.getData().forEach(singleData ->{
String xmlData =3D convertJson2Xml(singleData);
generateFopFactory(xmlData,templates, fopFactory);
});
=20
Thanks
=20
From: =E5=AD=A4=E7=8E=8B <[email protected]>=20
Sent: 04 June 2024 15:12
To: fop-users <[email protected]>
Subject: Re: RE: Fop batch generate report performance
=20
=20
Fop fop =3D =
fopFactory.newFop(MimeConstants.MIME_TIFF,fopFactory.newFOUserAgent(),out=
putStream);
=20
I am not sure as each records need to generate one file. So=EF=BC=8CHow =
do we use same Fop object for each outputstream?
=20
---Original---
From: "Simon Steiner"<[email protected] =
<mailto:[email protected]> >
Date: Tue, Jun 4, 2024 22:04 PM
To: "fop-users"<[email protected] =
<mailto:[email protected]> >;
Cc: "18224494266"<<[email protected] <mailto:[email protected]> >>;
Subject: RE: Fop batch generate report performance
=20
Hi,
=20
What about reusing the fopfactory, 1 per thread.
=20
Thanks
=20
From: =E5=AD=A4=E7=8E=8B <[email protected] =
<mailto:[email protected]> >=20
Sent: 04 June 2024 14:49
To: fop-users <[email protected] =
<mailto:[email protected]> >
Cc: 18224494266 <[email protected] <mailto:[email protected]> >
Subject: Fop batch generate report performance
=20
Dear Fop Docters,
=20
I am a student of Fop. May I ask some question for run the batch reports =
by Fop? Our company using the Fop to generate TIFF file. found the TIFF =
file cannot load font by lazy font. So, it will load all of the config =
folder fonts.
<renderer mime=3D"image/tiff">
<directory recursive =3D "true">/tmp/fonts</directory>
<compression>CCITT T.6</compression>
<endianness>LITTLE_ENDIAN</endianness>
</renderer>
=20
As our company all fonts more than 200M. So, when run the batch =
data(which records more 10000 each request). it mean one batch request =
will loop 10k times to create 10K Fop objects( =
fopFactory.newFop=EF=BC=88...=EF=BC=89=EF=BC=89. Each record of the =
batch need to load the font again. it lead to the generate is too slow.=20
=20
Could you give me some advise for new Fop object Repeatedly loading =
fonts for each record of one batch(one batch has 10k records need to =
loop).
Our found the Repeatedly loading fonts of Fop core code. As attachment =
picture "Fop custom load font code.PNG".
Our service code example as below.
@Override
public boolean fopGen(RequestData requestData, Templates templates) {
// one batch loop more than 10000 records.
requestData.getData().forEach(singleData ->{
String xmlData =3D convertJson2Xml(singleData);
generateFopFactory(xmlData,templates);
});
return true;
}
private void generateFopFactory(String xmlData, Templates templates){
FopFactory fopFactory =3D createFopFactory();
try(ByteArrayInputStream inputStream =3D new =
ByteArrayInputStream(xmlData.getBytes());
ByteArrayOutputStream outputStream =3D new =
ByteArrayOutputStream()){
Source src;
src =3D new StreamSource(inputStream);
Fop fop =3D =
fopFactory.newFop(MimeConstants.MIME_TIFF,fopFactory.newFOUserAgent(),out=
putStream);
Transformer transformer =3D templates.newTransformer();
final List<String> errList =3D new ArrayList<>();
transformer.setErrorListener(new FopErrorListener());
Result res =3D new SAXResult(fop.getDefaultHandler());
transformer.transform(src,res);
convert2File(outputStream);
} catch (IOException e) {
e.printStackTrace();
} catch (FOPException e) {
e.printStackTrace();
} catch (TransformerConfigurationException e) {
e.printStackTrace();
} catch (TransformerException e) {
e.printStackTrace();
}
}
=20
------=_NextPart_000_0014_01DAB692.91914C70
Content-Type: text/html;
charset="utf-8"
Content-Transfer-Encoding: quoted-printable
<html xmlns:v=3D"urn:schemas-microsoft-com:vml" =
xmlns:o=3D"urn:schemas-microsoft-com:office:office" =
xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" =
xmlns=3D"http://www.w3.org/TR/REC-html40"><head><meta =
http-equiv=3DContent-Type content=3D"text/html; charset=3Dutf-8"><meta =
name=3DGenerator content=3D"Microsoft Word 15 (filtered =
medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"MS Gothic";
panose-1:2 11 6 9 7 2 5 8 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
@font-face
{font-family:"\@MS Gothic";
panose-1:2 11 6 9 7 2 5 8 2 4;}
@font-face
{font-family:"\@SimSun";
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"PingFang SC";
panose-1:0 0 0 0 0 0 0 0 0 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;
mso-fareast-language:EN-GB;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext=3D"edit">
<o:idmap v:ext=3D"edit" data=3D"1" />
</o:shapelayout></xml><![endif]--></head><body lang=3DEN-GB =
link=3D"#0563C1" vlink=3D"#954F72"><div class=3DWordSection1><p =
class=3DMsoNormal><span =
style=3D'mso-fareast-language:EN-US'>Hi,<o:p></o:p></span></p><p =
class=3DMsoNormal><span =
style=3D'mso-fareast-language:EN-US'><o:p> </o:p></span></p><p =
class=3DMsoNormal><span style=3D'mso-fareast-language:EN-US'>What =
about:<br></span><span =
style=3D'font-size:8.5pt;font-family:SimSun'>FopFactory fopFactory =3D =
<i>createFopFactory</i>();<br><br><i><span =
style=3D'color:gray'> =
</span></i>requestData.getData().<b><span =
style=3D'color:red'>forEach</span></b>(singleData =
->{<br> String xmlData =3D =
convertJson2Xml(singleData);<br>  =
; generateFopFactory(xmlData,<span style=3D'color:#660E7A'>templates, =
fopFactory</span>);<br> });</span><span =
style=3D'mso-fareast-language:EN-US'><o:p></o:p></span></p><p =
class=3DMsoNormal><span =
style=3D'mso-fareast-language:EN-US'><o:p> </o:p></span></p><p =
class=3DMsoNormal><span =
style=3D'mso-fareast-language:EN-US'>Thanks<o:p></o:p></span></p><p =
class=3DMsoNormal><span =
style=3D'mso-fareast-language:EN-US'><o:p> </o:p></span></p><p =
class=3DMsoNormal><b><span lang=3DEN-US>From:</span></b><span =
lang=3DEN-US> </span><span style=3D'font-family:"MS =
Gothic"'>=E5=AD=A4=E7=8E=8B</span><span lang=3DEN-US> =
<[email protected]> <br><b>Sent:</b> 04 June 2024 =
15:12<br><b>To:</b> fop-users =
<[email protected]><br><b>Subject:</b> Re: RE: Fop =
batch generate report performance<o:p></o:p></span></p><p =
class=3DMsoNormal><o:p> </o:p></p><div><p =
class=3DMsoNormal><o:p> </o:p></p></div><div><pre><span =
style=3D'font-size:8.5pt;font-family:SimSun'>Fop fop =3D =
fopFactory.newFop(MimeConstants.<b><i><span =
style=3D'color:#660E7A'>MIME_TIFF</span></i></b>,fopFactory.newFOUserAgen=
t(),outputStream);</span><span =
style=3D'font-size:13.5pt'><o:p></o:p></span></pre></div><div><div><p =
class=3DMsoNormal><span =
style=3D'font-size:9.0pt'><o:p> </o:p></span></p></div><div><p =
class=3DMsoNormal><span style=3D'font-size:9.0pt'>I am not sure as each =
records need to generate one file. So</span><span =
style=3D'font-size:9.0pt;font-family:"MS Gothic"'>=EF=BC=8C</span><span =
style=3D'font-size:9.0pt'>How do we use same Fop object for each =
outputstream?<o:p></o:p></span></p></div><div><p class=3DMsoNormal><span =
style=3D'font-size:9.0pt'><o:p> </o:p></span></p></div><div><p =
class=3DMsoNormal><span =
style=3D'font-size:9.0pt'>---Original---<o:p></o:p></span></p></div><div>=
<div><p class=3DMsoNormal style=3D'background:#F0F0F0'><b><span =
style=3D'font-size:9.0pt;color:#212121'>From:</span></b><span =
style=3D'font-size:9.0pt;color:#212121'> "Simon Steiner"<<a =
href=3D"mailto:[email protected]">[email protected]</a>=
><o:p></o:p></span></p></div><div><p class=3DMsoNormal =
style=3D'background:#F0F0F0'><b><span =
style=3D'font-size:9.0pt;color:#212121'>Date:</span></b><span =
style=3D'font-size:9.0pt;color:#212121'> Tue, Jun 4, 2024 22:04 =
PM<o:p></o:p></span></p></div><div><p class=3DMsoNormal =
style=3D'background:#F0F0F0'><b><span =
style=3D'font-size:9.0pt;color:#212121'>To:</span></b><span =
style=3D'font-size:9.0pt;color:#212121'> "fop-users"<<a =
href=3D"mailto:[email protected]">[email protected]=
ache.org</a>>;<o:p></o:p></span></p></div><div><p class=3DMsoNormal =
style=3D'background:#F0F0F0'><b><span =
style=3D'font-size:9.0pt;color:#212121'>Cc:</span></b><span =
style=3D'font-size:9.0pt;color:#212121'> =
"18224494266"<<<a =
href=3D"mailto:[email protected]">[email protected]</a>>>;<o:p>=
</o:p></span></p></div><div><p class=3DMsoNormal =
style=3D'background:#F0F0F0'><b><span =
style=3D'font-size:9.0pt;color:#212121'>Subject:</span></b><span =
style=3D'font-size:9.0pt;color:#212121'> RE: Fop batch generate report =
performance<o:p></o:p></span></p></div></div><p =
class=3DMsoNormal><o:p> </o:p></p><div><p class=3DMsoNormal =
style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span =
style=3D'mso-fareast-language:EN-US'>Hi,</span><o:p></o:p></p><p =
class=3DMsoNormal =
style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span =
style=3D'mso-fareast-language:EN-US'> </span><o:p></o:p></p><p =
class=3DMsoNormal =
style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span =
style=3D'mso-fareast-language:EN-US'>What about reusing the fopfactory, =
1 per thread.</span><o:p></o:p></p><p class=3DMsoNormal =
style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span =
style=3D'mso-fareast-language:EN-US'> </span><o:p></o:p></p><p =
class=3DMsoNormal =
style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span =
style=3D'mso-fareast-language:EN-US'>Thanks</span><o:p></o:p></p><p =
class=3DMsoNormal =
style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span =
style=3D'mso-fareast-language:EN-US'> </span><o:p></o:p></p><p =
class=3DMsoNormal =
style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><b><span =
lang=3DEN-US style=3D'mso-fareast-language:ZH-CN'>From:</span></b><span =
lang=3DEN-US style=3D'mso-fareast-language:ZH-CN'> </span><span =
lang=3DZH-CN =
style=3D'font-family:SimSun;mso-fareast-language:ZH-CN'>=E5=AD=A4=E7=8E=8B=
</span><span lang=3DEN-US style=3D'mso-fareast-language:ZH-CN'> <<a =
href=3D"mailto:[email protected]">[email protected]</a>&g=
t; <br><b>Sent:</b> 04 June 2024 14:49<br><b>To:</b> fop-users <<a =
href=3D"mailto:[email protected]">[email protected]=
ache.org</a>><br><b>Cc:</b> 18224494266 <<a =
href=3D"mailto:[email protected]">[email protected]</a>><br><b>Sub=
ject:</b> Fop batch generate report performance</span><o:p></o:p></p><p =
class=3DMsoNormal =
style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p><=
/o:p></p><div><p class=3DMsoNormal =
style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Dear Fop =
Docters,<o:p></o:p></p></div><div><p class=3DMsoNormal =
style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p><=
/o:p></p></div><div><p class=3DMsoNormal =
style=3D'mso-margin-top-alt:auto;margin-bottom:12.0pt'>I am a student of =
Fop. May I ask some question for run the batch reports by Fop? Our =
company using the Fop to generate TIFF file. found the TIFF file cannot =
load font by lazy font. So, it will load all of the config folder =
fonts.<o:p></o:p></p><div><p class=3DMsoNormal =
style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><renderer=
mime=3D"image/tiff"><o:p></o:p></p></div><div><p =
class=3DMsoNormal =
style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> =
<directory recursive =3D =
"true">/tmp/fonts</directory><o:p></o:p></p></div><div=
><p class=3DMsoNormal =
style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> =
<compression>CCITT =
T.6</compression><o:p></o:p></p></div><div><p class=3DMsoNormal =
style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> =
<endianness>LITTLE_ENDIAN</endianness><o:p></o:p></p></div><d=
iv><p class=3DMsoNormal =
style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'></rendere=
r><o:p></o:p></p></div></div><div><p class=3DMsoNormal =
style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p><=
/o:p></p></div><div><p class=3DMsoNormal =
style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>As our =
company all fonts more than 200M. So, when run the batch data(which =
records more 10000 each request). it mean one batch request will loop =
10k times to create 10K Fop objects( fopFactory.newFop<span =
style=3D'font-family:"MS Gothic"'>=EF=BC=88</span>...<span =
style=3D'font-family:"MS Gothic"'>=EF=BC=89=EF=BC=89</span>. Each record =
of the batch need to load the font again. it lead to the generate is too =
slow. <o:p></o:p></p></div><div><p class=3DMsoNormal =
style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p><=
/o:p></p></div><div><p class=3DMsoNormal =
style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span =
style=3D'color:red'>Could you give me some advise for new Fop =
object Repeatedly loading fonts for each record of one batch(one =
batch has 10k records need to loop).</span><span =
style=3D'font-size:13.5pt;font-family:"PingFang =
SC",serif;color:#333333'><br></span>Our found the Repeatedly loading =
fonts of Fop core code. As attachment picture "Fop custom load font =
code.PNG".<br>Our service code example as =
below.<o:p></o:p></p><pre><span =
style=3D'font-size:8.5pt;font-family:SimSun;color:olive'>@Override<br></s=
pan><b><span =
style=3D'font-size:8.5pt;font-family:SimSun;color:navy'>public boolean =
</span></b><span =
style=3D'font-size:8.5pt;font-family:SimSun'>fopGen(RequestData =
requestData, Templates templates) {<br> <i><span =
style=3D'color:gray'>// one batch loop more than 10000 =
records.<br> </span></i>requestData.getData().<b><span =
style=3D'color:red'>forEach</span></b>(singleData =
->{<br> String xmlData =3D =
convertJson2Xml(singleData);<br>  =
; generateFopFactory(xmlData,<span =
style=3D'color:#660E7A'>templates</span>);<br> =
});<br> <b><span style=3D'color:navy'>return =
true</span></b>;<br>}<br><b><span style=3D'color:navy'>private void =
</span></b>generateFopFactory(String xmlData, Templates =
templates){<br> FopFactory fopFactory =3D =
<i>createFopFactory</i>();<br> <b><span =
style=3D'color:navy'>try</span></b>(ByteArrayInputStream inputStream =3D =
<b><span style=3D'color:navy'>new =
</span></b>ByteArrayInputStream(xmlData.getBytes());<br>  =
; ByteArrayOutputStream outputStream =3D =
<b><span style=3D'color:navy'>new =
</span></b>ByteArrayOutputStream()){<br> &nb=
sp; Source src;<br> src =
=3D <b><span style=3D'color:navy'>new =
</span></b>StreamSource(inputStream);<br> &n=
bsp; Fop fop =3D fopFactory.newFop(MimeConstants.<b><i><span =
style=3D'color:#660E7A'>MIME_TIFF</span></i></b>,fopFactory.newFOUserAgen=
t(),outputStream);<br> =
Transformer transformer =3D =
templates.newTransformer();<br> =
<b><span style=3D'color:navy'>final </span></b>List<String> =
errList =3D <b><span style=3D'color:navy'>new =
</span></b>ArrayList<>();<br> &n=
bsp; transformer.setErrorListener(<b><span style=3D'color:navy'>new =
</span></b>FopErrorListener());<br> &n=
bsp; Result res =3D <b><span style=3D'color:navy'>new =
</span></b>SAXResult(fop.getDefaultHandler());<br>  =
; =
transformer.transform(src,res);<br> &n=
bsp; convert2File(outputStream);<br> } <b><span =
style=3D'color:navy'>catch </span></b>(IOException e) =
{<br> =
e.printStackTrace();<br> } <b><span =
style=3D'color:navy'>catch </span></b>(FOPException e) =
{<br> =
e.printStackTrace();<br> } <b><span =
style=3D'color:navy'>catch </span></b>(TransformerConfigurationException =
e) {<br> =
e.printStackTrace();<br> } <b><span =
style=3D'color:navy'>catch </span></b>(TransformerException e) =
{<br> =
e.printStackTrace();<br> =
}<br>}</span><o:p></o:p></pre><pre><span =
style=3D'font-size:8.5pt;font-family:SimSun'> </span><o:p></o:p></pr=
e></div></div></div></div></body></html>
------=_NextPart_000_0014_01DAB692.91914C70--