RE: Should FLUTE File Aggregation I-D be a working group item?

Rod, all,

I support this being a WG item.

Further on the comments below, I think there are some advantages if we
can keep some commonality between the old and new FDT syntax for
physical aggregation as well as for logical aggregation. What I would
suggest is the following:

- in both cases the files are described in normal <File> elements, with
their own TOIs and a possible the 'Group' attribute
- the new <AggregateObject> aggregates a set of _objects_ (identified by
their TOIs) into a new aggregate object (which is given a TOI of its
own)

The <AggregateObject> element simply needs to define the list of TOIs
and their offsets into the aggregate object (their lengths and MIME
types etc. will already be in their corresponding <File> elements). We
don't really need multi-part MIME encoding of the aggregate object and
I'm not sure what value it adds.

This approach has a number of advantages:
- less new XML syntax needed!
- Receivers that do not support the <AggregateFile> element will at
least understand the <File> elements and could receive the files
individually if the sender chose to send them that way (i.e. if the
sender explicitly wanted to provide backwards compatibility for such
receivers, perhaps at some lower service level than for new receivers)
- We already have a mechanism for sending updated versions of a file: we
signal the same URI but a different TOI. The approach above preserves
this mechanism: The receiver can tell when a new aggregate object
contains some of the same objects as a previously received aggregate
object (same TOIs) - it can use the stored copies of these files to fill
in parts of the new object in advance (i.e. extract FEC source symbols
from the previously stored objects) and thus reduce the reception time
for the new object.

In fact, in cases where it is expected that many but not all users will
already have some of the files being aggregated, it may make sense for
the sender to send only FEC repair symbols for the aggregated file -
these can be combined at receivers with the source symbols obtained from
previously received files which have not changed in the new aggregated
object.

Some other minor comments:
- I don't think we need to worry about backwards compatibility with
FLUTE receivers compliant to the experimental version which did not have
element extensibility in the FDT XML - this was a mistake and the
'experimental' phase is exactly supposed to allow us to find and correct
such mistakes.
- the draft includes some text attaching semantics to TOI ordering -
something we removed from FLUTE in favour of FDT Instance IDs. We should
stick with the FLUTE principle that later version of a file is signaled
by using a different TOI in an FDT with a larger FDT Instance ID - it
doesn't matter whether the TOI value is greater or less than the
previous one.
- the draft mentions that one might only consider file aggregation when
the aggregate is less than a single source block - I think it is simpler
and has no disadvantages if you decouple the source blocking (which is a
feature of the FEC scheme chosen) from the file aggregation. If files
belong together in an aggregate then they should be put together and
then if the FEC code needs to split that data into blocks this is ok and
does not result in any worse performance so long as it correctly
interleaves the packets from the different blocks.

Regards,

Mark

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of
[email protected]
Sent: Tuesday, November 22, 2005 1:59 AM
To: [email protected]; [email protected]
Cc: [email protected]; [email protected]
Subject: RE: [Rmt] Should FLUTE File Aggregation I-D be a working group
item?

The thread reminded me of the RMT minutes which said something about
applications for object aggregation - I think "web sites" was questioned
in the meeting?

As for using object aggregation for websites - it's a good idea
(providing you have a way to pay for the process of doing it :)

Obviously for static content there are no big update issues. For more
dynamic content (regularly changing web sites), then it is likely that
some "complete site" update is less frequent than a "delta" to update
the changed parts. There is no requirement to use a physically
aggregated object for the delta. In such a case it would make sense to
logically group the large aggregated object and the small update file
(or files) - as the semantics of "all belong together" are preserved.

Also, we have the logical aggregation semantics. This groups files to
the same logical object without physically transporting them as one. For
large files (multiples of the source block size), the FEC benefits of
physical aggregation drop off so it is simpler to use logical
aggregation: although optimal configurations of FEC schemes differ, HD
movies >1GB are likely to fall into the "easily big enough to skip
physical aggregation" category.

There are other times when logical aggregation is desirable. 3GPP uses
the technique to allow (for multicast service announcements) a metadata
envelope (version etc.) to be grouped with its corresponding metadata
fragment (e.g. a media description) this way so the client would know
that these belong together and the content/MIME type indicates the
behaviour. Multipart MIME is also allowed to combine these two related
objects, but the release 6 specification does not detail exactly how all
metadata gets to the client and so this technique allows, e.g., repair
of the metadata over HTTP without a GET for a correctly received
envelope.

Otherwise, the selection between logical and physical aggregation is
pretty much implementation specific (i.e. application specific) - where
the preference for FEC performance and SW complexity is balanced
according to the use cases and environment.

Hope that helps (or at least helps more than it confuses :)

Cheers, Rod.



_______________________________________________
Rmt mailing list
[email protected]
https://www1.ietf.org/mailman/listinfo/rmt