Using Magick++ to convert large PDFs into individual PNGs

Lukas Studer <[email protected]> Tue, 9 Feb 2021 11:25:09 +0100
Newsgroups	gmane.comp.video.graphicsmagick.apis
Message-ID	<[email protected]>
--===============6633687366225618973==
Content-Type: multipart/alternative;
	boundary="------------18AC14F152FE2932F9DB174E"
Content-Language: en-US

--------------18AC14F152FE2932F9DB174E
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64

SGVsbG8sCgoKRmlyc3Qgb2YgYWxsLCBJJ20gbmV3IHRvIEdyYXBoaWNzTWFnaWNrIGFuZCB0
aGUgbWFuYWdlbWVudCB0aGVyZW9mLCBzbyBJCmFtIG5vdCBxdWl0ZSBzdXJlIGlmIHRoaXMg
aXMgdGhlIGNvcnJlY3QgbWFpbGluZyBsaXN0LgoKSSdtIGFtIHVzaW5nIHRoZSBNYWdpY2sr
KyBBUEkgdG8gcmVhZCBsYXJnZSBQREYgZmlsZXMgKGJhbGxwYXJrIG9mIGFib3V0CjIwMDAg
dG8gMzAwMCBwYWdlcywgc2l6ZSBvZiAxMDBNQiBhbmQgdXB3YXJkcykgYW5kIHN0b3JlIGVh
Y2ggcGFnZSBpbiBhCmRhdGFiYXNlIChub3Qgc2hvd24gaW4gc25pcHBldCkuIFdoaWxlIGRv
aW5nIHNvIEkgcnVuIGludG8gcmVzb3VyY2UKaXNzdWVzLCBuYW1lbHkgbm90IGVub3VnaCBS
QU0sIHdoZW4gcmVhZGluZyB0aGUgUERGIGZpbGUgcmVzdWx0aW5nIGluCmJhc2ljYWxseSwg
b3Igd2hhdCBmZWVscyBsaWtlLCBhbiBpbmZpbml0ZSBydW50aW1lLgoKTXkgcXVlc3Rpb24g
bm93IGJvaWxzIGRvd24gdG8sIHdoYXQgaXMgdGhlIHByb3BlciBBUEkgZm9yIHN1Y2ggYSB1
c2UKY2FzZSwgaS5lLiBob3cgZG9lcyBvbmUgaGFuZGxlIHN1Y2ggYmlnIFBERiBmaWxlcyB1
c2luZyB0aGUgTWFnaWNrKysKQVBJPyBJcyB0aGVyZSBhIHdheSB0byBkbyB0aGlzIGluIGJh
dGNoZXMgb3IgaW4gYSBzdHJlYW1lZCBmYXNoaW9uLAp3aGVyZSBJIHJlYWQgcGFnZSBieSBw
YWdlIGFuZCBkaXJlY3RseSBzdG9yZSB0aGVtPwoKSSBrbm93IG9mIHRoZSBzeW50YXggaW52
b2x2aW5nIHRoZSBpbmRleE9mIG9wZXJhdG9yIHRvIG9ubHkgcmVhZCBhIHJhbmdlCm9mIHBh
Z2VzLCBlLmcuICJteUZpbGUucGRmWzAtMTAwXSIuIEkgd2FzIGhvcGluZyB0byBzb2x2ZSB0
aGUgaXNzdWUKZGlyZWN0bHkgdmlhIHRoZSBNYWdpY2srKyBBUEkuCgoKU25pcHBldDoKCnN0
ZDo6c3RyaW5nIERhdGFiYXNlOjppbXBvcnRQREYoY29uc3Qgc3RkOjpzdHJpbmcmIHBkZlBh
dGgsIGNvbnN0CnN0ZDo6c3RyaW5nJiBuYW1lKQp7wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
wqDCoArCoMKgwqDCoMKgwqDCoCBzdGQ6Omlmc3RyZWFtIGlucHV0RmlsZShwZGZQYXRoKTsK
wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoArCoMKgwqDC
oMKgwqDCoCBpZiAoIWlucHV0RmlsZS5nb29kKCkpCsKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoCByZXR1cm4gIiI7CgrCoMKgwqDCoMKgwqDCoCAvLyBnZXQgY3VycmVudCBkYXRl
IGFuZCB0aW1lIGluIGlzbyBmb3JtYXQKwqDCoMKgwqDCoMKgwqAgc3RkOjpsb2NhbGU6Omds
b2JhbChzdGQ6OmxvY2FsZSgiIikpOwrCoMKgwqDCoMKgwqDCoCBzdGQ6OmNocm9ubzo6c3lz
dGVtX2Nsb2NrOjp0aW1lX3BvaW50IG5vdyA9CnN0ZDo6Y2hyb25vOjpzeXN0ZW1fY2xvY2s6
Om5vdygpOwrCoMKgwqDCoMKgwqDCoCBzdGQ6OnRpbWVfdCB0ID0gc3RkOjpjaHJvbm86OnN5
c3RlbV9jbG9jazo6dG9fdGltZV90KG5vdyk7CgrCoMKgwqDCoMKgwqDCoCBjaGFyIGRhdGVC
dWZmWzEwMF07CsKgwqDCoMKgwqDCoMKgIHN0ZDo6c3RyZnRpbWUoZGF0ZUJ1ZmYsIHNpemVv
ZihkYXRlQnVmZiksICIlRiIsIHN0ZDo6bG9jYWx0aW1lKCZ0KSk7CsKgwqDCoMKgwqDCoMKg
CgrCoMKgwqDCoMKgwqDCoCBNYWdpY2s6OkluaXRpYWxpemVNYWdpY2soIiIpOwrCoMKgwqDC
oMKgwqDCoCBzdGQ6Omxpc3Q8TWFnaWNrOjpJbWFnZT4gaW1hZ2VzOwrCoMKgwqDCoMKgwqDC
oCBNYWdpY2s6OnJlYWRJbWFnZXMoJmltYWdlcywgcGRmUGF0aCk7CgrCoMKgwqDCoMKgwqDC
oCBzaXplX3QgaTsKwqDCoMKgwqDCoMKgwqAgdW5zaWduZWQgaW50IGogPSAwOwrCoMKgwqDC
oMKgwqDCoCBzdGQ6Omxpc3Q8TWFnaWNrOjpJbWFnZT46Oml0ZXJhdG9yIGltYWdlOwrCoMKg
wqDCoMKgwqDCoCBmb3IgKGltYWdlID0gaW1hZ2VzLmJlZ2luKCksIGkgPSAwOyBpbWFnZSAh
PSBpbWFnZXMuZW5kKCk7CmltYWdlKyssIGkrKywgaisrKSB7CsKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoCBNYWdpY2s6OkJsb2IgaW1hZ2VEYXRhOwrCoMKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqAgaW1hZ2UtPm1hZ2ljaygiUE5HIik7CsKgwqDCoMKgwqDCoMKgwqDC
oMKgwqDCoMKgwqDCoCBpbWFnZS0+d3JpdGUoJmltYWdlRGF0YSk7CsKgwqDCoMKgwqDCoMKg
wqDCoMKgwqDCoMKgwqDCoCB0aGlzLT5hZGRQYWdlKCIiLCBpbWFnZURhdGEuYmFzZTY0KCks
IGltYWdlLT5yb3dzKCksCmltYWdlLT5jb2x1bW5zKCksIGosIDAsIGkrMSwgaW1hZ2VzLnNp
emUoKSwgMCk7CsKgwqDCoMKgwqDCoMKgIH3CoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg
CsKgCsKgwqDCoMKgwqDCoMKgIHJldHVybiBwZGZQYXRoOwp9CgoKVGhhbmtzIGFuZCBDaGVl
cnMsCkx1a2FzCgo=
--------------18AC14F152FE2932F9DB174E
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: 8bit

<html>
  <head>

    <meta http-equiv="content-type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <p>Hello,</p>
    <p><br>
    </p>
    <p>First of all, I'm new to GraphicsMagick and the management
      thereof, so I am not quite sure if this is the correct mailing
      list.</p>
    <p>I'm am using the Magick++ API to read large PDF files (ballpark
      of about 2000 to 3000 pages, size of 100MB and upwards) and store
      each page in a database (not shown in snippet). While doing so I
      run into resource issues, namely not enough RAM, when reading the
      PDF file resulting in basically, or what feels like, an infinite
      runtime.</p>
    <p>My question now boils down to, what is the proper API for such a
      use case, i.e. how does one handle such big PDF files using the
      Magick++ API? Is there a way to do this in batches or in a
      streamed fashion, where I read page by page and directly store
      them?</p>
    <p>I know of the syntax involving the indexOf operator to only read
      a range of pages, e.g. "myFile.pdf[0-100]". I was hoping to solve
      the issue directly via the Magick++ API.<br>
    </p>
    <p><br>
    </p>
    <p>Snippet:<br>
    </p>
    <p><font size="+1"><tt>std::string Database::importPDF(const
          std::string&amp; pdfPath, const std::string&amp; name)</tt><tt><br>
        </tt><tt>{               </tt><tt><br>
        </tt><tt>        std::ifstream inputFile(pdfPath);</tt><tt><br>
        </tt><tt>                        </tt><tt><br>
        </tt><tt>        if (!inputFile.good())</tt><tt><br>
        </tt><tt>                return "";</tt><tt><br>
        </tt><tt><br>
        </tt><tt>        // get current date and time in iso format</tt><tt><br>
        </tt><tt>        std::locale::global(std::locale(""));</tt><tt><br>
        </tt><tt>        std::chrono::system_clock::time_point now =
          std::chrono::system_clock::now();</tt><tt><br>
        </tt><tt>        std::time_t t =
          std::chrono::system_clock::to_time_t(now);</tt><tt><br>
        </tt><tt><br>
        </tt><tt>        char dateBuff[100];</tt><tt><br>
        </tt><tt>        std::strftime(dateBuff, sizeof(dateBuff), "%F",
          std::localtime(&amp;t));</tt><tt><br>
        </tt><tt>        </tt><tt><br>
        </tt><tt><br>
        </tt><tt>        Magick::InitializeMagick("");</tt><tt><br>
        </tt><tt>        std::list&lt;Magick::Image&gt; images;</tt><tt><br>
        </tt><tt>        Magick::readImages(&amp;images, pdfPath);</tt><tt><br>
        </tt><tt><br>
        </tt><tt>        size_t i;</tt><tt><br>
        </tt><tt>        unsigned int j = 0;</tt><tt><br>
        </tt><tt>        std::list&lt;Magick::Image&gt;::iterator image;</tt><tt><br>
        </tt><tt>        for (image = images.begin(), i = 0; image !=
          images.end(); image++, i++, j++) {</tt><tt><br>
        </tt><tt>                Magick::Blob imageData;</tt><tt><br>
        </tt><tt>                image-&gt;magick("PNG");</tt><tt><br>
        </tt><tt>                image-&gt;write(&amp;imageData);</tt><tt><br>
        </tt><tt>                this-&gt;addPage("",
          imageData.base64(), image-&gt;rows(), image-&gt;columns(), j,
          0, i+1, images.size(), 0);</tt><tt><br>
        </tt><tt>        }               </tt><tt><br>
        </tt><tt> </tt><tt><br>
        </tt><tt>        return pdfPath;</tt><tt><br>
        </tt><tt>}</tt></font><br>
      <br>
    </p>
    <p><br>
    </p>
    <p>Thanks and Cheers,<br>
      Lukas<br>
    </p>
  </body>
</html>

--------------18AC14F152FE2932F9DB174E--


--===============6633687366225618973==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline


--===============6633687366225618973==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Graphicsmagick-apis mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/graphicsmagick-apis

--===============6633687366225618973==--