RE: Should FLUTE File Aggregation I-D be a working group item?

Magnus, all,

I agree with your comments below that a new aggregation scheme must
offer advantages over simply zipping the files. I think such advantages
do exist.

Some of the requirements that should be met by an aggregation scheme -
which illustrate these advantages - are as follows:

- backwards-compatibility - at least source symbols sent for the files
being aggregated should be able to be interpreted by receivers which do
not support the aggregation scheme
- incremental update - as I mentioned in my other email, it should be
possible for repair symbols for the aggregate object to be combined with
source symbols which are already known (perhaps from receipt of an
earlier version of the aggregate which contained some of the same files)
- partial reception - if sufficient data has been received to recover
part of the aggregate, then it should be possible to extract the files
which are contained within the recovered part
- single file transmission - it should still be possible to send the
files individually, in particular for updates

So I think there is a question for the draft authors on whether they are
happy to take on these requirements in making the draft a working group
item - it may result in the solution being rather different from that
presently proposed.

...Mark

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of
Magnus Westerlund
Sent: Tuesday, November 29, 2005 12:17 AM
To: [email protected]
Cc: [email protected]; [email protected]
Subject: Re: [Rmt] Should FLUTE File Aggregation I-D be a working group
item?

Hi,

To clarify my position and say that I don't care about specific
solutions.

Yes I am talking about the physically aggregated objects in previous 
emails. I think I share the same concerns that Joerg Ott raised. There 
is no meaning to reinvent an exact duplication of tar or zip at FLUTE 
level. If one should do this in flute rather than simply write a 
recommendation that applications that have many small files should use a

aggregated file container, then I think we need to provide some clear 
benefits, for example extraction of files from partially recovered 
aggregates. Simplified or more efficient recovery and handling of 
updates of the files part of the aggregate.

Simply using multi-party MIME does not really gain anything compared to 
tar or zip, then perhaps somewhat clear rules on how URI for files are 
constructed. For example multi-party MIME requires the capability to 
search the aggregate from start to end to find delimiters. This 
operation could be seriously hampered or even produce errors if the file

is partial. In addition it is missing any integrity mechanism that I 
think is important if one tries to use partially repaired aggregates.

My proposal is to define a physical aggregation mechanism that primarily

defines how one builds the source block from multiple files and then 
allows the receiver to perform the reverse operation to extract them 
from that aggregate also in cases when decode has not been completely 
successful. For large burst errors the loss for an aggregate may result 
in loss of one or several objects within the aggregate but still have 
parts of the aggregate for which all source symbols has been received.

So if one can find a way around the issues with backwards compatible 
FDT, sure no problem. However my view on this is that if we introduce a 
aggregation mechanism it will any way result in all receiver to support 
it, some further required extensions of the FDT will not cost us any 
interoperability.


Cheers

Magnus Westerlund

Multimedia Technologies, Ericsson Research EAB/TVA/A
----------------------------------------------------------------------
Ericsson AB                | Phone +46 8 4048287
Torshamsgatan 23           | Fax   +46 8 7575550
S-164 80 Stockholm, Sweden | mailto: [email protected]

_______________________________________________
Rmt mailing list
[email protected]
https://www1.ietf.org/mailman/listinfo/rmt