Re: TextIO.inputN dramatically worse than TextIO.input

On Wed, Feb 4, 2015 at 8:34 PM, Michael Norrish
<[email protected]> wrote:
> In the file at
>
> https://github.com/HOL-Theorem-Prover/HOL/blob/40158b75c83806e2901edf0037677582098ec5a3/tools/quote-filter/mlton-quote-filter.sml
>
> there is some code for implementing a simple Unix-style filter, based on some
> logic in an mllex lexer (that’s what’s hiding in the structure “filter”).
>
> On lines 13 and 21 I set the lexer up with an input function that is supposed to
> return some of the input.  For the interactive version (line 21), using
> TextIO.input makes sense because you want to echo (filtered) user input back as
> soon as possible.  But in the non-interactive version, I initially thought I’d
> use TextIO.inputN.
>
> I was surprised to find that it actually performed 4 times worse on an input
> file of size 10MB, taking about 80s instead of 20s.
>
> The fact that either version is taking 20s is pretty painful in itself (a
> Poly/ML version of this program does the same input in about 4s).
>
> I’m running on
>
>>    Linux telemachus 3.2.0-75-generic #110-Ubuntu SMP Tue Dec 16 19:11:55 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
>
> with mlton version 20100608.
>
> Should I expect to see this discrepancy?  And what is the point of inputN if so?

In general, I might expect TextIO.inputN to be ever so slightly slower
than TextIO.input.  The latter simply returns whatever remains in the
input buffer and only performs actual I/O when the input buffer is
empty.  The former requires that we do best effort to get exactly n
characters, which can mean extra copying (if we have less than n
characters left in the input buffer) and less efficient I/O (because
we only request n characters, rather than the desired buffer size).
mllex generated code tries to use 1024 as the request size, whereas
MLton's TextIO.input defaults to using 4096 as the default request
size.

However, I couldn't reproduce the discrepancy on my MacOSX system:

[mtf@graywolf quote-filter]$ uname -a
Darwin graywolf.local 14.1.0 Darwin Kernel Version 14.1.0: Mon Dec 22
23:10:38 PST 2014; root:xnu-2782.10.72~2/RELEASE_X86_64 x86_64
[mtf@graywolf quote-filter]$ /usr/bin/time ./unquote.20100608.input
big-input big-output.20100608.input
        2.79 real         1.46 user         1.31 sys
[mtf@graywolf quote-filter]$ /usr/bin/time ./unquote.20100608.inputN
big-input big-output.20100608.inputN
        2.81 real         1.47 user         1.32 sys
[mtf@graywolf quote-filter]$ /usr/bin/time ./unquote.20130715.input
big-input big-output.20130715.input
        2.85 real         1.48 user         1.36 sys
[mtf@graywolf quote-filter]$ /usr/bin/time ./unquote.20130715.inputN
big-input big-output.20130715.inputN
        2.79 real         1.46 user         1.32 sys

big-input is a 9M file generated by repeatedly concatenating the
sample input file from the quote-filter directory.

I can try later on my linux system.

As for the point of inputN, I think it makes a little more sense as
BinIO.inputN, where one might use it to extract the exact bytes
corresponding to fields of some header data, rather than using
BinIO.input and various Slice operations (plus handling the case when
a field spans the boundary between the bytes returned by two
successive BinIO.input calls).

To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].

------------------------------------------------------------------------------
Dive into the World of Parallel Programming. The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net/
_______________________________________________
MLton-user mailing list
[email protected]; [email protected]
https://lists.sourceforge.net/lists/listinfo/mlton-user