Re: Strings all the way down?

On Sat, May 02, 2009 at 11:15:10AM -0700, Chris Babcock wrote:

> > just wondering if you're seeing the quoted i/o size limit - can you
> > post a snippet of code where this problem shows up?
> 
> The 256k limit explains subsequent behavior as well as the initial
> problem...

old docs anyway; currenty in crm114_config.h we have:

//    define the "block ratio" of how of a memory data window we're
//    willing to suck in from a minion process before we block on
//    sucking; the un-sucked part just waits in the minion's stdout
//    buffer (and causes the minion to block on output).  Normally a
//    factor of 2 (1/4th of the size of a full memory window, or 2
//    megabytes in the default configuraton) is sufficient.
#define SYSCALL_WINDOW_RATIO 2

and then in crm_expr_syscall.c

      if (keep_proc == 1 || async_mode == 1)
        {
          //   we're in either 'keep' 'async' mode.  Set nonblocking mode, then
          //   read it once; then put it back in regular mode.
          //fcntl (from_minion[0], F_SETFL, O_NONBLOCK);
          //      usleep (timeout);
          charsread = read (from_minion[0],
                            &outbuf[done],
                            (data_window_size >> SYSCALL_WINDOW_RATIO));

ie 2MB block as you've actually seen; you can try SYSCALL_WINDOW_RATIO 1
and see if that's better/enough for your case without the {liaf}.

> syscall <keep> (::)(:in:)(:stat:) /convert control.ps pnm:-/

here pstopnm(1) - ie gs(1) - is twice as fast as convert(1).

> Thank you! Now to see if there's any time savings left...

you're welcome :)

-- 
paolo

------------------------------------------------------------------------------
Register Now & Save for Velocity, the Web Performance & Operations 
Conference from O'Reilly Media. Velocity features a full day of 
expert-led, hands-on workshops and two days of sessions from industry 
leaders in dedicated Performance & Operations tracks. Use code vel09scf 
and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf