Re: Re: can enbd server run in kernel?

On Sat, 2004-11-27 at 21:53, Peter T. Breuer wrote:
> "Alse sprach Ming Zhang:"
> > so i think this fits my guess. uncached mmap will not bring much
> > benefits here. i wonder if you could do one more test with cached mmap,
> 
> The problem is that this measurement does not show something one is
> certainly interested in - cpu loading. 
yes, cpu load is a dynamic thing. can not be captured by strace, or hard
to tell from. and frankly, we only save some memcpy, which can show up
only when u have huge data traffic here.

> 
> I would have expected time to be affected too, but it seems not to be 
> obvious.
> 
> > ignore the data consistency first. i guess with that, there should be
> > some difference with fast gigabit network and large size data transfer.
> 
> Well, I have introduced map caching and used a look ahead of 1MB, and
> it helps, but not greatly.  At the sizes of this test, the disk writes
> are all cached in VMS anyway and measure very fast, as fast as writes to
> memory. When I pushed mmaps up to 1MB cached, they took about 220us to
> make, 10-20 times as long as for 4KB maps. 
> 
can this look ahead to be larger? like ~100MB?

> Unfortunately it seems (from man pages) as though msync really is needed
> after writes to the map, although only with ms_invalidate, nothing more.
> And I can't yet get it to work on only regions of the map. I am
> investigating.
> 
> With 4KB requests, and 1MB maps cached (and all msyncs failing, but it
> doesn't matter since there is only one thread), I get:
> 
> % time     seconds  usecs/call     calls    errors syscall
>   0.42    0.108365         223       486           old_mmap
>   0.36    0.091915         190       485           munmap
>   0.11    0.029165           6      4927           write
>   0.11    0.028450           6      4972           read
>   0.07    0.017984           4      4921      4921 msync
>   4.20    1.072929         118      9074           send
>   0.38    0.097121           8     11944           recv
> 
> The test prveviously looked like this:
> 
> With mmapping on:
> 
> % time     seconds  usecs/call     calls    errors syscall
>   0.18    0.046989          10      4925           old_mmap
>   0.13    0.035787           7      4925           munmap
>   0.11    0.030343           6      4931           write
>   0.10    0.026622           5      4980           read
>   2.46    0.657184         132      4988           send
>   0.29    0.078830           8      9910           recv
> 
en, i do not realize that this mmap took so many time. so strange. :P i
blindly guess it needs to lock some pages already in system?

if strace can not do this well, i guess we have to use something like
oprofile or kernprof.

> With mmapping off:
> 
> % time     seconds  usecs/call     calls    errors syscall
>   0.29    0.082280           8      9886           write
>   0.17    0.049714           7      7042           read
>   3.66    1.049559         115      9096           send
>   0.26    0.075423           6     11985           recv
> 
> 
> Peter