Re: Bittorrent and filesystem API / integration

--- In [email protected], Jesus Cea <jcea@a...> wrote:
> Nick Johnson wrote:
> > No, that's why I stated that you should check the mtime of the bitmap 
> > versus the files it concerns before starting. As long as the program 
> > ensures it flushes its buffers for the files before writing the
updated 
> > bitmap, you know that if the bitmap is newer than the files, it is
valid.
> 
> Yes. You must do "fsync" everywhere. Performance sucks :-p

If you would do that for every bit you update, yes. A better
approach would be:
- Make the transaction size for the modifications larger. So, if the
bittorrent protocol tries to stream data more contiguous if possible,
reducing seeks on the drives of both clients, one can cheaply combine
the transaction. It costs hardly time to write 1 MB instead of 512
bytes. if fully contiguous, flushing the transactions to disk every
minute, or earlier if the load on the disk subsystem allows for it, 
would be 'good enough' in case of a power faillure. Buffering only 1
MB of data is not 'too much' by todays memory footprints.
One could even go further, and think about virtual memory. MMapping
the to-be-written chunks and lazyly flushing them to disk would make
the kernel / disk queing pick the 'best time' to write them to disk.
 
- Make sure your filesystem supports 'application defined attributes',
metadata on files, or attributes that are deleted and moved with a
file. The latter property is an important advantage over mirroring in
'.foo' dirs and other hidden files. 
I'd like to see something like that implemented for subversion - but
would make it not cross-client compatible - damn 'file access and
fileformat' API's!

> Perhaps better to update the bitmap not everytime you complete a 
> part, but every minute or so.

Something that would be very cool would be a harddrive that can
write-cache things persistenly, using flash (if doesn't wear out too
fast) and/or RAM that is powered by a capacitor, allowing it to 
write those bytes on powerloss to disk, while the disk is already
spinning down. Writing to small sectors etc can be a lot faster, no
immediate seeks needed.

Mark-Jan






 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/BitTorrent/

<*> To unsubscribe from this group, send an email to:
    [email protected]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/