Re: Bittorrent and filesystem API / integration

Justin Cormack wrote:
>>
>>
>>Hi,
>>
>>Checking the integrity of a file before sharing it is logical
>>behaviour of BitTorrent. But, on the other hand, it is also kind of
>>annoying, having to wait until the file is fully scanned. It also
>>costs a lot of disk bandwidth, a device that has to deal already with
>>quite a lot of 'contention' for seeking opeations on it.
> 
> 
> Technically a BT client doesnt have to do this, it could treat startup
> like having an empty file, and then as it checks blocks it could send
> have messages to the peers. There is a cost as the have messages are
> substantially larger than a bitmap however.
> 
> Another way to save time would be some way to detect sparse files quickly
> (one way is to check if a block is all zeros and if the expected hash
> is not the hash of a zero block, which can be precomputed. This would
> save some CPU time when scanning files.

It's the disk usage/time that matters, not the CPU time.

>>So, I was thinking: would it be cool to find a filesystem and API,
>>that is more advanced than the basic open/read/write/append/close
>>behaviour we seen unchanged since the 70's ? 
>>
>>Things like:
>>- Integrity checks on chunks of a file, signatures of an application 
>>  that has written part of a file, and kernel guarantees that only you 
>>  have access to a file, until you 'unlock' the file for usage by other 
>>  applications.
> 
> 
> This is very dubious, as disks are unreliable and guaranteeing this in
> any error situation is very difficult.

Not that unreliable I hope. If a disk fails, it usually doesn't 'return' 
success.
A FS that could store data checksums/hashes would be very nice for all 
kinds of caching though.

>>- Transactional file modifications in chunks, queing up transactions,
>>  prioritizing them, optimizing a filesystem for the anticipated size
>>  of a file before even a single byte is written.
> 
> 
> Some file systems do support preallocation for files (forget offhand
> which ones). Asynchronous io gives you queued transactions.

Only simple transactions though.


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/BitTorrent/

<*> To unsubscribe from this group, send an email to:
    [email protected]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/