Re: [MLton] follow-up [cost of _export]

On Tue, Dec 9, 2014 at 10:59 AM, Bernard Berthomieu <[email protected]> wrote:
> On 12/08/14 22:42, Bernard Berthomieu wrote:
>> I will try the patch asap and let you know the
>> improvements on my use case.
>
> Could not wait :-) .
>
> The experiment consists of parsing a 16MB xml file.
> There are 1 011 136 calls from C to SML.
>
>
> gc-summary before the patch:
>
> GC type        time ms     number          bytes          bytes/sec
> -------------    -------    -------    --------------- ---------------
> copying         13,720        775      2,123,511,788 154,774,913
> mark-compact          0          0                  0               -
> minor              0          0                  0 -
> total time: 27,121 ms
> total GC time: 20,487 ms (75.5%)
> max pause time: 313 ms
> total bytes allocated: 14,275,417,936 bytes
> max bytes live: 22,243,940 bytes
> max heap size: 178,012,160 bytes
> max stack size: 10,384 bytes
> ...
>
> The total time drops to 8s with "fixed-heap 1G", but rises to
> 500s (99% of which in gc) with "copy-generational-ratio 8.0"
> (default for my applications) with a huge amount of minor gc.

I'm not exactly sure why the generational GC is so problematic.
Possibly due to the fact that all of the created threads almost
certainly need to have their stack resized and a resized stack is
pre-tenured in the old generation.  Thus, the minor GCs miss much of
the actual garbage.

> gc-summary after the patch:
>
> GC type        time ms     number          bytes          bytes/sec
> -------------    -------    -------    --------------- ---------------
> copying            289         12         48,134,248 166,554,491
> mark-compact          0          0                  0               -
> minor              0          0                  0 -
> total time: 2,362 ms
> total GC time: 519 ms (22.0%)
> max pause time: 200 ms
> total bytes allocated: 225,872,036 bytes
> max bytes live: 20,448,692 bytes
> max heap size: 163,635,200 bytes
> max stack size: 8,776 bytes
> ...
>
> Total time drops to 1.87s with "fixed-heap 1G" and stays
> unchanged with "copy-generational-ratio 8.0" (no minor gc
> is triggered).
>
> A cheap _export opens up new perpectives :-)

Great!  I'll merge the patch into the master branch shortly.

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk