Re: Using Magick++ to convert large PDFs into individual PNGs

--===============3741594913980453840==
Content-Type: multipart/alternative;
	boundary="------------8D29DEB012D5F747B73E8C02"
Content-Language: en-US

--------------8D29DEB012D5F747B73E8C02
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64

SGkgQm9iLAoKCkkgbWFuYWdlZCB0byBnZXQgYSBjaHVua2VkLCBwYXJhbGxlbCBpbXBvcnQg
b2YgbGFyZ2UgUERGIGZpbGVzIHdvcmtpbmcKdXNpbmcgT3Blbk1QIGFuZCB0aGUgaW5kZXgt
b2YgbGlrZSBvcGVyYXRvciBpbiB0aGUgZmlsZSBuYW1lIHNwZWNpZmllcgphcyB5b3Ugc3Vn
Z2VzdGVkLgoKSSBhbHNvIG92ZXJsb2FkZWQgdGhlIHJlYWRJbWFnZXMgZnVuY3Rpb24gYWRk
aW5nIGEgcmVhZCBhcmd1bWVudHMgc3RydWN0CnRvIHNldCBzb21lIGF0dHJpYnV0ZXMgaW4g
dGhlIGltYWdlIGluZm8gc3RydWN0IChzZWUgY29kZSBiZWxvdykuIFVzaW5nCnRoaXMgSSBj
YW4gbm93IGFsc28gb3ZlcnJpZGUgdGhlIGRlbnNpdHkgZm9yIHRoZSBpbXBvcnQuIEkgbm90
aWNlZCB0aGlzCndhcyBhbHNvIGZsYWdnZWQgd2l0aCAiRklYTUUiLgoKdHlwZWRlZiBzdHJ1
Y3QgUmVhZEFyZ3MKe8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoArCoMKgwqDCoMKg
wqDCoCBpbnQgc3ViaW1hZ2UgPQotMTvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgCsKg
wqDCoMKgwqDCoMKgIGludCBzdWJyYW5nZSA9IC0xOwrCoMKgwqDCoMKgwqDCoCBjb25zdCBz
dGQ6OnN0cmluZyBkZW5zaXR5ID0gIiI7Cn0gUmVhZEFyZ3M7Cgp0ZW1wbGF0ZSA8Y2xhc3Mg
Q29udGFpbmVyPgp2b2lkIHJlYWRJbWFnZXMoIENvbnRhaW5lciAqc2VxdWVuY2VfLArCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBjb25zdCBzdGQ6OnN0cmluZwomaW1hZ2VT
cGVjXyzCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgCsKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFJlYWRBcmdzIGFyZ3MgPSB7fSkgewrCoMKgwqDC
oMKgwqDCoCBNYWdpY2tMaWI6OkltYWdlSW5mbyAqaW1hZ2VJbmZvID0KTWFnaWNrTGliOjpD
bG9uZUltYWdlSW5mbygwKTvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgCgrCoMKgwqDCoMKgwqDCoCBpbWFnZVNwZWNfLmNvcHkoIGltYWdlSW5mby0+
ZmlsZW5hbWUsIE1heFRleHRFeHRlbnQtMQopO8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoArCoMKgwqDCoMKgwqDCoCBpbWFn
ZUluZm8tPmZpbGVuYW1lWyBpbWFnZVNwZWNfLmxlbmd0aCgpIF0gPQowO8KgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoAoKwqDCoMKgwqDCoMKgwqAgaWYgKGFyZ3Muc3ViaW1hZ2UgPj0gMCkK
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGltYWdlSW5mby0+c3ViaW1hZ2UgPSBh
cmdzLnN1YmltYWdlOwoKwqDCoMKgwqDCoMKgwqAgaWYgKGFyZ3Muc3VicmFuZ2UgPj0KMCnC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqAKwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGltYWdl
SW5mby0+c3VicmFuZ2UgPSBhcmdzLnN1YnJhbmdlOwoKwqDCoMKgwqDCoMKgwqAgaWYgKGFy
Z3MuZGVuc2l0eSAhPSAiIikKe8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgCsKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoCBpbWFnZUluZm8tPmRlbnNpdHkgPSAoY2hhciAqKQptYWxsb2MoYXJncy5k
ZW5zaXR5Lmxlbmd0aCgpKTvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAK
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHN0ZDo6c3RyY3B5KGltYWdlSW5mby0+
ZGVuc2l0eSwgYXJncy5kZW5zaXR5LmNfc3RyKCkpOwrCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqAgaW1hZ2VJbmZvLT5kZW5zaXR5WyBhcmdzLmRlbnNpdHkubGVuZ3RoKCkgXSA9
IDA7CsKgwqDCoMKgwqDCoMKgIH0KCsKgwqDCoMKgwqDCoMKgIE1hZ2lja0xpYjo6RXhjZXB0
aW9uSW5mbyBleGNlcHRpb25JbmZvOwrCoMKgwqDCoMKgwqDCoCBNYWdpY2tMaWI6OkdldEV4
Y2VwdGlvbkluZm8oICZleGNlcHRpb25JbmZvICk7CsKgwqDCoMKgwqDCoMKgIE1hZ2lja0xp
Yjo6SW1hZ2UqIGltYWdlcyA9wqAgTWFnaWNrTGliOjpSZWFkSW1hZ2UoIGltYWdlSW5mbywK
JmV4Y2VwdGlvbkluZm8gKTsKwqDCoMKgwqDCoMKgwqAgTWFnaWNrTGliOjpEZXN0cm95SW1h
Z2VJbmZvKGltYWdlSW5mbyk7CsKgwqDCoMKgwqDCoMKgIGluc2VydEltYWdlcyggc2VxdWVu
Y2VfLCBpbWFnZXMpOwrCoMKgwqDCoMKgwqDCoCB0aHJvd0V4Y2VwdGlvbiggZXhjZXB0aW9u
SW5mbyApOwp9CgpJIGRpZCBub3RpY2UgdGhvdWdoIHRoYXQgc2V0dGluZyB0aGUgInN1Ymlt
YWdlIiBhbmQgInN1YnJhbmdlIgphdHRyaWJ1dGVzIGluIEltYWdlSW5mbyB3YXMgc2lnbmlm
aWNhbnRseSBzbG93ZXIgdGhhbiB1c2luZyB0aGUgZmlsZQpuYW1lIHNwZWNpZmllci4gRG8g
eW91IGhhdmUgYW55IGlkZWEgYXMgdG8gd2h5IHRoaXMgbWlnaHQgYmU/CgpPbiBhIHNsaWdo
dGx5IGRpZmZlcmVudCB0b3BpYzogSXQgYXBwZWFycyB0aGF0IHRoZSB3cml0aW5nIG9mIFBE
RiBmaWxlcwpzdWZmZXJzIHRoZSBzYW1lIGxpbWl0YXRpb25zIGFzIHRoZSByZWFkaW5nIHRo
ZXJlb2YuIFdyaXRpbmcgbGFyZ2UKZG9jdW1lbnRzIGNhdXNlcyB0aGUgUkFNIHRvIG92ZXJm
bG93IGNyYXNoaW5nIHRoZSBhcHBsaWNhdGlvbi4gRG8geW91CmhhdmUgYW55IGluc2lnaHRz
IG9uIGhvdyB0byBrZWVwIHRoZSBSQU0gdXNhZ2UgaW4gY2hlY2sgd2hpbGUgd3JpdGluZwp0
aGUgUERGIGZpbGVzPyBJcyB0aGVyZSBhIHdheSB0byB3cml0ZSB0aGVtIGluIGNodW5rcz8g
RnVydGhlciwgSSBkb24ndApxdWl0ZSB1bmRlcnN0YW5kIHdoeSB0aGlzIGhhcHBlbnMgaW4g
Z2VuZXJhbC4gSG93IGlzIGl0IHRoYXQgYSBQREYgZmlsZQpvZiBhYm91dCAxMDBNQiBjYW4g
b3ZlcmZsb3cgbXkgUkFNIG9mIGFib3V0IDY0R0IgKHdoZW4KaW1wb3J0aW5nL2V4cG9ydGlu
ZyBhdCAzMDAgRFBJKS4gSXMgdGhlcmUgc29tZXRoaW5nIEknbSBkb2luZyBob3JyaWJseQp3
cm9uZz8gV2h5IGlzIGl0IGJsb2F0ZWQgdGhpcyBtdWNoPwoKClRoYW5rcyBmb3IgeW91ciBo
ZWxwIQoKQ2hlZXJzLApMdWthcwoKT24gMTAuMDIuMjEgMTU6MDgsIEJvYiBGcmllc2VuaGFo
biB3cm90ZToKPiBPbiBUdWUsIDkgRmViIDIwMjEsIEx1a2FzIFN0dWRlciB3cm90ZToKPj4K
Pj4gSSB3aWxsIGhhdmUgYSBsb29rIGF0IHlvdXIgc3VnZ2VzdGlvbnMgYW5kIHNlZSB3aGlj
aCBzb2x1dGlvbiBzdWl0cyBteQo+PiBuZWVkcyBiZXN0Lgo+Cj4gSSBmb3Jnb3Qgb2Ygc29t
ZXRoaW5nIHdoaWNoIG1heSBoZWxwIHNvbHZlIHRoZSBwcm9ibGVtIGZvciB5b3UuwqAgVGhl
Cj4gc3RhcnRpbmcgcGFnZSBvZmZzZXQgYW5kIHJhbmdlIG1heSBiZSBzcGVjaWZpZWQgYXMg
cGFydCBvZiB0aGUgaW5wdXQKPiBmaWxlIG5hbWUuwqAgVGhpcyBtZWFucyB0aGF0IHlvdSBj
YW4gYWxyZWFkeSB1c2UgaXQgdG8gcmVxdWVzdCBhIHJhbmdlCj4gb2YgcGFnZXMgdXNpbmcg
dGhlIGV4aXN0aW5nIGltcGxlbWVudGF0aW9uLgo+Cj4gU28sIHlvdSBjYW4gcmVxdWVzdCAn
ZmlsZS5wZGZbMC05XScsIGFuZCB0aGVuICdmaWxlLnBkZlsxMC0xOV0nLCBldGMuCj4gaW4g
b3JkZXIgdG8ga2VlcCB0aGUgbnVtYmVyIG9mIHBhZ2VzIHJlcXVlc3RlZCBhdCBvbmUgdGlt
ZSBkb3duIHRvIGEKPiBtYW5hZ2VhYmxlIGxldmVsLgo+Cj4gQm9iCg==
--------------8D29DEB012D5F747B73E8C02
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: 8bit

<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <p>Hi Bob,</p>
    <p><br>
    </p>
    <p>I managed to get a chunked, parallel import of large PDF files
      working using OpenMP and the index-of like operator in the file
      name specifier as you suggested.</p>
    <p>I also overloaded the <font size="+1"><tt>readImages</tt></font>
      function adding a read arguments struct to set some attributes in
      the image info struct (see code below). Using this I can now also
      override the density for the import. I noticed this was also
      flagged with "FIXME".<br>
    </p>
    <p><tt>typedef struct ReadArgs
        {                                                                     
      </tt><tt><br>
      </tt><tt>        int subimage =
        -1;                                                                    
      </tt><tt><br>
      </tt><tt>        int subrange = -1;</tt><tt><br>
      </tt><tt>        const std::string density = "";</tt><tt><br>
      </tt><tt>} ReadArgs;</tt><tt><br>
      </tt><tt><br>
      </tt><tt>template &lt;class Container&gt;</tt><tt><br>
      </tt><tt>void readImages( Container *sequence_,</tt><tt><br>
      </tt><tt>                 const std::string
        &amp;imageSpec_,                                               
      </tt><tt><br>
      </tt><tt>                 ReadArgs args = {}) {</tt><tt><br>
      </tt><tt>        MagickLib::ImageInfo *imageInfo =
        MagickLib::CloneImageInfo(0);                        </tt><tt><br>
      </tt><tt><br>
      </tt><tt>        imageSpec_.copy( imageInfo-&gt;filename,
        MaxTextExtent-1 );                               </tt><tt><br>
      </tt><tt>        imageInfo-&gt;filename[ imageSpec_.length() ] =
        0;                                        </tt><tt><br>
      </tt><tt><br>
      </tt><tt>        if (args.subimage &gt;= 0)</tt><tt><br>
      </tt><tt>                imageInfo-&gt;subimage = args.subimage;</tt><tt><br>
      </tt><tt><br>
      </tt><tt>        if (args.subrange &gt;=
        0)                                                               
      </tt><tt><br>
      </tt><tt>                imageInfo-&gt;subrange = args.subrange;</tt><tt><br>
      </tt><tt><br>
      </tt><tt>        if (args.density != "")
        {                                                              </tt><tt><br>
      </tt><tt>                imageInfo-&gt;density = (char *)
        malloc(args.density.length());                   </tt><tt><br>
      </tt><tt>                std::strcpy(imageInfo-&gt;density,
        args.density.c_str());</tt><tt><br>
      </tt><tt>                imageInfo-&gt;density[
        args.density.length() ] = 0;</tt><tt><br>
      </tt><tt>        }</tt><tt><br>
      </tt><tt><br>
      </tt><tt>        MagickLib::ExceptionInfo exceptionInfo;</tt><tt><br>
      </tt><tt>        MagickLib::GetExceptionInfo( &amp;exceptionInfo
        );</tt><tt><br>
      </tt><tt>        MagickLib::Image* images =  MagickLib::ReadImage(
        imageInfo, &amp;exceptionInfo );</tt><tt><br>
      </tt><tt>        MagickLib::DestroyImageInfo(imageInfo);</tt><tt><br>
      </tt><tt>        insertImages( sequence_, images);</tt><tt><br>
      </tt><tt>        throwException( exceptionInfo );</tt><tt><br>
      </tt><tt>}</tt><br>
      <br>
    </p>
    <p>I did notice though that setting the "subimage" and "subrange"
      attributes in ImageInfo was significantly slower than using the
      file name specifier. Do you have any idea as to why this might be?<br>
    </p>
    <p>On a slightly different topic: It appears that the writing of PDF
      files suffers the same limitations as the reading thereof. Writing
      large documents causes the RAM to overflow crashing the
      application. Do you have any insights on how to keep the RAM usage
      in check while writing the PDF files? Is there a way to write them
      in chunks? Further, I don't quite understand why this happens in
      general. How is it that a PDF file of about 100MB can overflow my
      RAM of about 64GB (when importing/exporting at 300 DPI). Is there
      something I'm doing horribly wrong? Why is it bloated this much?<br>
    </p>
    <p><br>
    </p>
    <p>Thanks for your help!</p>
    <p>Cheers,<br>
      Lukas<br>
    </p>
    <div class="moz-cite-prefix">On 10.02.21 15:08, Bob Friesenhahn
      wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:[email protected]">On
      Tue, 9 Feb 2021, Lukas Studer wrote:
      <br>
      <blockquote type="cite">
        <br>
        I will have a look at your suggestions and see which solution
        suits my
        <br>
        needs best.
        <br>
      </blockquote>
      <br>
      I forgot of something which may help solve the problem for you. 
      The starting page offset and range may be specified as part of the
      input file name.  This means that you can already use it to
      request a range of pages using the existing implementation.
      <br>
      <br>
      So, you can request 'file.pdf[0-9]', and then 'file.pdf[10-19]',
      etc. in order to keep the number of pages requested at one time
      down to a manageable level.
      <br>
      <br>
      Bob
      <br>
    </blockquote>
  </body>
</html>

--------------8D29DEB012D5F747B73E8C02--


--===============3741594913980453840==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline


--===============3741594913980453840==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Graphicsmagick-apis mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/graphicsmagick-apis

--===============3741594913980453840==--