Re: [Fwd: Re: [Fwd: [CPS-devel] directorystorage and ext3?]]

On Saturday 14 May 2005 14:57, Jean-Marc Orliaguet wrote:
> Hi!
>
> according to the benchmarks I'd done so far, ext3 does not seem to scale
> well after 3 or 4GB of data (corresponding to ~ 400.000 small files in
> the storage)
>
> I've simply uploaded documents:
> - 1000 x 2MB image attachment
> - 1000 x 80Kb pdf file (this also includes a transformation from pdf to
> text)
>
> and measured the time that it takes to upload each document:
>
> 1000x 2MB image
> - http://www.medic.chalmers.se/~jmo/dokarkiv/perf-2.pdf
>
> + 1000 x 80kb pdf file
> - http://www.medic.chalmers.se/~jmo/dokarkiv/perf-3.pdf

Interesting graphs. thanks for sharing.

A benchmark performing non-stop writes may not be a good indicator of 
performance under a mixed read/write or read-heavy workload. During a write 
transaction DS writes the new data into the journal directory. This is quick, 
to reduce latency of the write transaction. When sufficient write 
transactions have completed it asynchronously moves them all from the journal 
into the main data directory in a batch. This is known as 'journal flushing', 
and is relatively slow.

Under sustained heavy write pressure the journal flushing process becomes the 
bottleneck. Eventually one write transaction has to wait for the previous 
journal flush to complete (to prevent the journal getting too large), causing 
spikes exactly like I see in that graph. That seriously slows down 
performance, but Ive only ever seen it happen in benchmarks.

I wouldnt expect to see those spikes if the benchmark was to repeatedly upload 
1 document then read 10 old ones. 

> It seems that directory storage does a transaction flush every ten
> document or so, which knocks ext3 out.

There are some parameters in the [journal] section of the configuration file 
which control journal flushing. Controlling when it starts flushing a batch 
of transactions, and the number of outstanding batches before a write 
transaction gets blocked. Im sure they could benefit from some tweaking to 
improve performance in this write-heavy operating mode. 



-------------------------------------------------------
This SF.Net email is sponsored by Oracle Space Sweepstakes
Want to be the first software developer in space?
Enter now for the Oracle Space Sweepstakes!
http://ads.osdn.com/?ad_id=7393&alloc_id=16281&op=click