Ted's concerns about wastefulness (Accept-* problem)

Ted wrote on 19 Jan 1999

> First, my apologies for the delay in commenting on the work Bill and
> Graham have put forward; second, my thanks to both of them for taking
> on the hardest problem this group has left to solve.

Alas, now I have to apologize for my own delayed reply.. Sorry!

I'll quote a big chunk of text, then reply to it.
 
> Having named it hard, I would like to take a crack at defining the
> problem and discussing what methods there are for solving it.  (In
> the text below, I use the term "grouping" to talk about features
> which are associated with each other, but do not fully describe
> the features of a resource or the capabilities of a device.  It's
> not a particularly great phrase, and replacement suggestions are
> welcome).
> 
> The Problem:
> 
> When there are multiple, static representations of a particular
> resource, the features which describe the differences between the
> representations are bounded and the method by which the resources can
> be distinguished are pretty clear.  In our current architecture the
> method for doing so requires, however, that the features which do not
> distinguish between them be repeated in every group.  This means that
> a representation which can be printed on a4 paper, in 16 shades of
> grey, using postscript level 2, using a times-roman 12 point font must
> be distinguished from a representation which can be printed on
> na-letter, in 16 shades of grey, using postscript level 2, using
> courier 12 point font.  (Note, by the way, that the groupings in the
> example imply that you can fit things on to na-letter only by changing
> fonts; you cannot simply vary both the paper and the font).  The
> constant features cannot be left out, because the devices need to
> compare their capabilities against them, but the waste of repeating
> them goes up enormously with the number of representations; with
> dynamic representations, this gets even worse.  Having some way to
> reference the constant features without repeating them would obviously
> be a win.  
> 
> The same is true on the flip side; there are a lot capabilities within
> devices, most of them constant.  Having some way to reference the
> constant set without repeating them is also a big win, especially
> where bandwidth is low.
> 
> 
> The Example in many of our minds:
> 
> In HTTP, sending a full set of Accept-* headers got to be so wasteful
> that many devices stopped using that method for negotiation and
> shifted to User-Agent or other methods (this was, of course, only part
> of the problem, and a full treatment would include the "Accept */*"
> problem and other things).  In a sense, User-Agent became a feature
> grouping identifier; it wasn't a good one, though, because it often
> either implied capabilities which had been disabled or did not include
> extensions.
> 
> Possibilities:
> 
> As Bill and the CCPP work implies, you can use a URI that points to a
> constant grouping.  That URI can be used as a token, for situations in
> which both negotiating parties understand the feature grouping it
> implies.  The URI token will not, sadly, be known to both parties in
> all situations.  If the URI gives a pointer to a location with a
> canonical representation of the features it implies, chances of
> success go up, but it will not always be possible to retrieve the
> representation and disaggregate the features.  (Both for the case in
> which a device cannot use the URI scheme and for the case in which the
> canonical representation of the resource is not available).  A
> traditional URI won't help in the "multiple inclusion" aspect of the
> problem, because you need to assign the groupings dynamically.  We
> could steal the work of the mhtml work and assign an "internal" URI to
> the set of constant items as part of the message, then use that
> identifier as an internal reference.

> The primary problem with the URI method is update; it requires systems
> to constantly keep up with new URIs, some of which will represent the
> same constellation of features.  On the cell phone side, for example,
> Qualcomm dhas branded the phone I have with 5 different company names
> that I am aware of, and I would bet each would require references to
> *their* URI space.

This is part of what I had in mind in when I said that nothing in the
proposal should interfere with the ability of web caches to cache
feature set resources. I believe that the ability to cache the
resources received from URLs will allow industrial-strength servers to
avoid having to do many over-the-net lookups of device capability
sets: they will have seen a PlusZoomerCorp model 44 several times in the
last hour, and the expiration time for its capability expression will
be at least a day, so they'll be able to use the local copy of its
capabilities. It seems to me that this caching behavior could achieve
efficiency within a small constant factor (1.5?) of the efficiency to
be expected from a central registry of standard feature sets.

Some web caching systems are already addressing the situation of
identical content at different URLs: they hash the contents of
everything they cache, notice the duplicate content, and store only
one copy. So, even if vendors do the relabelling that you're concerned
about, the overhead of 5 relabels for Qualcomm phones could be be as
low as 4 extra URL entries in the cache hash table -- a few hundred
bytes at most.

> Any other possible solutions that scale to the extent we need?

Given the existing HTTP caching mechanism, and assuming that device
vendors would have the good sense to use reasonable expiration times
on their capability resources (at least a day) I'd expect an
industrial-strength Web server serving thousands of hits per hour
would hit its internal cache 95+% of the time, and lesser Web servers
would hit the nearest external cache 95+% of the time. (Rationale:
I figure that 95% of the usage would come from the most-common 
500 models or so.)

Incidentally, I'm not just showing a touching faith in human nature
when I conjecture that the vendors wouldn't mess up caching. A vendor
who messes up caching causes problems not just for the system in
general, but for his customers in particular: they're the ones who
have to wait for extra net round trips.

  Bill Newman
  [email protected]