Maximum string size?
Chris Angelico <[email protected]> Tue, 11 Jul 2017 00:48:49 +1000
| Newsgroups | gmane.comp.lang.pike.user |
|---|---|
| Message-ID | <CAPTjJmrdLrNu+6c7qaXVRj3ma+oVVWKbVtveGQaSUANfA7R=0A@mail.gmail.com> |
Okay, I know this is sloppy and inefficient code, but bear with me :)
I'm building up an animated GIF by constructing frames in PPM format
internally, and then doing this:
array(string) animation = allocate(frame_count);
//populate array with data
Process.run(({"ffmpeg", "-y", "-f", "image2pipe", "-i", "-", filename
+ ".gif"}),
(["stdin": animation * ""]));
Or rather i was, until I found that a 2GB string sometimes terminates
Pike (or throws an exception, not sure) with the message "String too
long". This happened consistently after rendering frame 331, which is
the point at which the big stdin string hits 2GB. However, I have
managed to generate larger strings in testing. I've now changed to
piping the data using a secondary thread, avoiding the creation of the
temporary string, but am curious as to what this limit actually is.
And it's not a RAM size limit; "/usr/bin/time -v" shows the max RSS to
have been no more than a few gig, and I've easily exceeded that on
other occasions.
The trouble is that this is perfectly consistent when I do the full
job, but doesn't seem to replicate when I try to short-hand the
process - and the full job takes hours. (I just made a mistake and
threw away six hours of processing, and it was about a third of the
way through.) Where can I go looking to find the cause of this?
ChrisA