RE: Fop batch generate report performance
"Simon Steiner" <[email protected]> Tue, 4 Jun 2024 15:03:54 +0100
| Newsgroups | gmane.text.xml.fop.user |
|---|---|
| Message-ID | <[email protected]> |
------=_NextPart_000_0009_01DAB690.6BB4D550 Content-Type: text/plain; charset="iso-2022-jp" Content-Transfer-Encoding: 7bit Hi, What about reusing the fopfactory, 1 per thread. Thanks From: 孤王瘢雹 <[email protected]> Sent: 04 June 2024 14:49 To: fop-users <[email protected]> Cc: 18224494266 <[email protected]> Subject: Fop batch generate report performance Dear Fop Docters, I am a student of Fop. May I ask some question for run the batch reports by Fop? Our company using the Fop to generate TIFF file. found the TIFF file cannot load font by lazy font. So, it will load all of the config folder fonts. <renderer mime="image/tiff"> <directory recursive = "true">/tmp/fonts</directory> <compression>CCITT T.6</compression> <endianness>LITTLE_ENDIAN</endianness> </renderer> As our company all fonts more than 200M. So, when run the batch data(which records more 10000 each request). it mean one batch request will loop 10k times to create 10K Fop objects( fopFactory.newFop（...））. Each record of the batch need to load the font again. it lead to the generate is too slow. Could you give me some advise for new Fop object Repeatedly loading fonts for each record of one batch(one batch has 10k records need to loop). Our found the Repeatedly loading fonts of Fop core code. As attachment picture "Fop custom load font code.PNG". Our service code example as below. @Override public boolean fopGen(RequestData requestData, Templates templates) { // one batch loop more than 10000 records. requestData.getData().forEach(singleData ->{ String xmlData = convertJson2Xml(singleData); generateFopFactory(xmlData,templates); }); return true; } private void generateFopFactory(String xmlData, Templates templates){ FopFactory fopFactory = createFopFactory(); try(ByteArrayInputStream inputStream = new ByteArrayInputStream(xmlData. getBytes()); ByteArrayOutputStream outputStream = new ByteArrayOutputStream()){ Source src; src = new StreamSource(inputStream); Fop fop = fopFactory.newFop(MimeConstants.MIME_TIFF,fopFactory.newFOUserAgent(),output Stream); Transformer transformer = templates.newTransformer(); final List<String> errList = new ArrayList<>(); transformer.setErrorListener(new FopErrorListener()); Result res = new SAXResult(fop.getDefaultHandler()); transformer.transform(src,res); convert2File(outputStream); } catch (IOException e) { e.printStackTrace(); } catch (FOPException e) { e.printStackTrace(); } catch (TransformerConfigurationException e) { e.printStackTrace(); } catch (TransformerException e) { e.printStackTrace(); } } ------=_NextPart_000_0009_01DAB690.6BB4D550 Content-Type: text/html; charset="iso-2022-jp" Content-Transfer-Encoding: quoted-printable <html xmlns:v=3D"urn:schemas-microsoft-com:vml" = xmlns:o=3D"urn:schemas-microsoft-com:office:office" = xmlns:w=3D"urn:schemas-microsoft-com:office:word" = xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" = xmlns=3D"http://www.w3.org/TR/REC-html40"><head><meta = http-equiv=3DContent-Type content=3D"text/html; = charset=3Diso-2022-jp"><meta name=3DGenerator content=3D"Microsoft Word = 15 (filtered medium)"><style><!-- /* Font Definitions */ @font-face {font-family:SimSun; panose-1:2 1 6 0 3 1 1 1 1 1;} @font-face {font-family:"MS Gothic"; panose-1:2 11 6 9 7 2 5 8 2 4;} @font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} @font-face {font-family:Consolas; panose-1:2 11 6 9 2 2 4 3 2 4;} @font-face {font-family:"\@MS Gothic"; panose-1:2 11 6 9 7 2 5 8 2 4;} @font-face {font-family:"\@SimSun"; panose-1:2 1 6 0 3 1 1 1 1 1;} @font-face {font-family:"PingFang SC"; panose-1:0 0 0 0 0 0 0 0 0 0;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; margin-bottom:.0001pt; font-size:11.0pt; font-family:"Calibri",sans-serif;} pre {mso-style-priority:99; mso-style-link:"HTML Preformatted Char"; margin:0cm; margin-bottom:.0001pt; font-size:10.0pt; font-family:"Courier New";} span.HTMLPreformattedChar {mso-style-name:"HTML Preformatted Char"; mso-style-priority:99; mso-style-link:"HTML Preformatted"; font-family:Consolas; mso-fareast-language:EN-GB;} span.EmailStyle20 {mso-style-type:personal-reply; font-family:"Calibri",sans-serif; color:windowtext;} .MsoChpDefault {mso-style-type:export-only; font-family:"Calibri",sans-serif; mso-fareast-language:EN-US;} @page WordSection1 {size:612.0pt 792.0pt; margin:72.0pt 72.0pt 72.0pt 72.0pt;} div.WordSection1 {page:WordSection1;} --></style><!--[if gte mso 9]><xml> <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" /> </xml><![endif]--><!--[if gte mso 9]><xml> <o:shapelayout v:ext=3D"edit"> <o:idmap v:ext=3D"edit" data=3D"1" /> </o:shapelayout></xml><![endif]--></head><body lang=3DEN-GB = link=3D"#0563C1" vlink=3D"#954F72"><div class=3DWordSection1><p = class=3DMsoNormal><span = style=3D'mso-fareast-language:EN-US'>Hi,<o:p></o:p></span></p><p = class=3DMsoNormal><span = style=3D'mso-fareast-language:EN-US'><o:p> </o:p></span></p><p = class=3DMsoNormal><span style=3D'mso-fareast-language:EN-US'>What about = reusing the fopfactory, 1 per thread.<o:p></o:p></span></p><p = class=3DMsoNormal><span = style=3D'mso-fareast-language:EN-US'><o:p> </o:p></span></p><p = class=3DMsoNormal><span = style=3D'mso-fareast-language:EN-US'>Thanks<o:p></o:p></span></p><p = class=3DMsoNormal><span = style=3D'mso-fareast-language:EN-US'><o:p> </o:p></span></p><p = class=3DMsoNormal><b><span lang=3DEN-US = style=3D'mso-fareast-language:ZH-CN'>From:</span></b><span lang=3DEN-US = style=3D'mso-fareast-language:ZH-CN'> </span><span lang=3DZH-CN = style=3D'font-family:SimSun;mso-fareast-language:ZH-CN'>=1B$B8I2&=1B(B</s= pan><span lang=3DEN-US style=3D'mso-fareast-language:ZH-CN'> = <[email protected]> <br><b>Sent:</b> 04 June 2024 = 14:49<br><b>To:</b> fop-users = <[email protected]><br><b>Cc:</b> 18224494266 = <[email protected]><br><b>Subject:</b> Fop batch generate report = performance<o:p></o:p></span></p><p = class=3DMsoNormal><o:p> </o:p></p><div><p class=3DMsoNormal>Dear = Fop Docters,<o:p></o:p></p></div><div><p = class=3DMsoNormal><o:p> </o:p></p></div><div><p class=3DMsoNormal = style=3D'margin-bottom:12.0pt'>I am a student of Fop. May I ask some = question for run the batch reports by Fop? Our company using the Fop to = generate TIFF file. found the TIFF file cannot load font by lazy font. = So, it will load all of the config folder fonts.<o:p></o:p></p><div><p = class=3DMsoNormal><renderer = mime=3D"image/tiff"><o:p></o:p></p></div><div><p = class=3DMsoNormal> <directory recursive =3D = "true">/tmp/fonts</directory><o:p></o:p></p></div><div= ><p class=3DMsoNormal> <compression>CCITT = T.6</compression><o:p></o:p></p></div><div><p = class=3DMsoNormal> = <endianness>LITTLE_ENDIAN</endianness><o:p></o:p></p></div><d= iv><p = class=3DMsoNormal></renderer><o:p></o:p></p></div></div><div><p = class=3DMsoNormal><o:p> </o:p></p></div><div><p = class=3DMsoNormal>As our company all fonts more than 200M. So, when run = the batch data(which records more 10000 each request). it mean one batch = request will loop 10k times to create 10K Fop objects( = fopFactory.newFop<span style=3D'font-family:"MS = Gothic"'>=1B$B!J=1B(B</span>...<span style=3D'font-family:"MS = Gothic"'>=1B$B!K!K=1B(B</span>. Each record of the batch need to load = the font again. it lead to the generate is too = slow. <o:p></o:p></p></div><div><p = class=3DMsoNormal><o:p> </o:p></p></div><div><p = class=3DMsoNormal><span style=3D'color:red'>Could you give me some = advise for new Fop object Repeatedly loading fonts for each record = of one batch(one batch has 10k records need to loop).</span><span = style=3D'font-size:13.5pt;font-family:"PingFang = SC",serif;color:#333333'><br></span>Our found the Repeatedly loading = fonts of Fop core code. As attachment picture "Fop custom load font = code.PNG".<br>Our service code example as = below.<o:p></o:p></p><pre><span = style=3D'font-size:8.5pt;font-family:SimSun;color:olive'>@Override<br></s= pan><b><span = style=3D'font-size:8.5pt;font-family:SimSun;color:navy'>public boolean = </span></b><span = style=3D'font-size:8.5pt;font-family:SimSun'>fopGen(RequestData = requestData, Templates templates) {<br> <i><span = style=3D'color:gray'>// one batch loop more than 10000 = records.<br> </span></i>requestData.getData().<b><span = style=3D'color:red'>forEach</span></b>(singleData = ->{<br> String xmlData =3D = convertJson2Xml(singleData);<br>  = ; generateFopFactory(xmlData,<span = style=3D'color:#660E7A'>templates</span>);<br> = });<br> <b><span style=3D'color:navy'>return = true</span></b>;<br>}<br><b><span style=3D'color:navy'>private void = </span></b>generateFopFactory(String xmlData, Templates = templates){<br> FopFactory fopFactory =3D = <i>createFopFactory</i>();<br> <b><span = style=3D'color:navy'>try</span></b>(ByteArrayInputStream inputStream =3D = <b><span style=3D'color:navy'>new = </span></b>ByteArrayInputStream(xmlData.getBytes());<br>  = ; ByteArrayOutputStream outputStream =3D = <b><span style=3D'color:navy'>new = </span></b>ByteArrayOutputStream()){<br> &nb= sp; Source src;<br> src = =3D <b><span style=3D'color:navy'>new = </span></b>StreamSource(inputStream);<br> &n= bsp; Fop fop =3D fopFactory.newFop(MimeConstants.<b><i><span = style=3D'color:#660E7A'>MIME_TIFF</span></i></b>,fopFactory.newFOUserAgen= t(),outputStream);<br> = Transformer transformer =3D = templates.newTransformer();<br> = <b><span style=3D'color:navy'>final </span></b>List<String> = errList =3D <b><span style=3D'color:navy'>new = </span></b>ArrayList<>();<br> &n= bsp; transformer.setErrorListener(<b><span style=3D'color:navy'>new = </span></b>FopErrorListener());<br> &n= bsp; Result res =3D <b><span style=3D'color:navy'>new = </span></b>SAXResult(fop.getDefaultHandler());<br>  = ; = transformer.transform(src,res);<br> &n= bsp; convert2File(outputStream);<br> } <b><span = style=3D'color:navy'>catch </span></b>(IOException e) = {<br> = e.printStackTrace();<br> } <b><span = style=3D'color:navy'>catch </span></b>(FOPException e) = {<br> = e.printStackTrace();<br> } <b><span = style=3D'color:navy'>catch </span></b>(TransformerConfigurationException = e) {<br> = e.printStackTrace();<br> } <b><span = style=3D'color:navy'>catch </span></b>(TransformerException e) = {<br> = e.printStackTrace();<br> = }<br>}<o:p></o:p></span></pre><pre><span = style=3D'font-size:8.5pt;font-family:SimSun'><o:p> </o:p></span></pr= e></div></div></body></html> ------=_NextPart_000_0009_01DAB690.6BB4D550--