Re: [Fwd: Re: CAP-12-a: Stored queries still?]

Doug replied on 09/03/2003 06:07:24 PM:
> > We never reached a clear concensus on the issue of stored queries in 
> > CAP.  The thread just stopped in mid-January 2003 and I still question 

> > the usefulness of them in CAP.
> 
> Is what Bruce means, it it does not end until he agrees. There was
> a debate - two of them over the last two years.

Check the archives and you'll see that its not only I who questioned the 
usefulness of the feature as we have designed CAP.    Go ahead, I'll 
wait...

Recently I asked for examples of where its really useful and really has 
benefit versus the cost and complexity to CS implementations.  Of the 2 
people on the list who wanted to keep them (Doug & Andrea), neither 
responded.  Why is that?  Instead of showing a benefit for them, you try 
to deflect the issue now by saying already been covered.  In fact, it was 
not resolved.

We debated the initial idea way back when it was proposed as a solution to 
"thin" clients like PDAs and the like.  The idea was to reduce a CUAs need 
to recraft a common query and just rely on the CS keeping it around. 
Common activities like checking for alarms, searching for new 'stuff', etc 
were all proposed as desireable cases that would be used for this feature 
and there would be lots of saved bandwidth and cycles. 

In taking a practical look at them in recent drafts I question their 
actual usefulness given our query lingua. 

Since there is no way for any client ('fat' or 'thin') to create a stored 
query that is really beneficial and reusable I question the need to have 
them as a mandatory CS feature.  The query language has NO way for 
crafting a query thats reusable like "Find me the alarms that go off in 
the next 15 minutes" or "What entries are on my calendar for today". Thats 
because we have no concept of being able to wildcards; the query MUST use 
exact comparisons to _explicit_ values. 

This hardly makes the stored querys reusable; the CUA will have to update 
them with new _explicit_ values on some regular basis.  Not much savings 
there that I can see and infact it may be more of a burden (ie: extra 
network traffic to recreate the stored query) than a benefit.

> Which is irrelevant as there ARE queries that can be saved
> without wild cards. 

Yes there are; _any_ query can be stored.  However they do NOT fulfull the 
original intent of the stored query feature nor does storing them in the 
CS any strong benefit to anyone compared to the cost of implementation.

>                      No place in CAP does it say that
> you must be able to do stored queries based on time or wild cards.

You are missing the point or trying to deflect it.

The intent of stored queries was to make it easier for 'thin' clients to 
keep smaller footprints by allowing them to not have to (re)craft commonly 
used queries again and again; they relied on the CS to keep the query and 
involk it when the CUA needed it.  This is a great concept but we failed 
to design our query language to allow for it to be _actually_ useful.

Until now noone has provided any kind of examples that show why we MUST 
have this as part of ALL CS implemenations.  The CAP drafts make an 
implicit requirement that CS implement stored queries and I think this is 
wrong to do, especially given the lack of any demonstratable benefit.

Now that you've finally got some examples, lets take a look at them to see 
if they meet the original intent and compare any benefits to the cost of 
saving queries...

>    QUERYID:Common
>    SELECT VEVENT FROM VEGANDA WHERE STATE() = "UNPROCESSED"
>    SELECT VTODO FROM VEGANDA WHERE STATE() = "UNPROCESSED"

Yes, this is a common kind of query ("Find me 'new stuff'"). 

However there is nothing complex here that would require any CUA ('fat' or 
'thin') to take up that much extra space or cycles to send.  It can simply 
be a hardcoded string that any CUA simply puts into the payload.  Of 
course you can bloat this query out to be explicit for all kinds of 
components explicitly (and thus miss unprocessed content) or you can take 
the more compact form and use:

SELECT * FROM VEGANDA WHERE STATE() = "UNPROCESSED"

The savings in doing this is 51 octets compared to sending just:

QUERYID:Common

Umm, not any real savings for any kind of CUA that I can see.

>    QUERYID:STUFF
>    SELECT VEVENT FROM VAGENDA WHERE STATE() = "DELETED"
>    SELECT VTODO FROM VAGENDA WHERE STATE() = "DELETED"

Is this a common query?  Find me the deleted entries? 

Perhaps if you were building some kind of replication or sync engine on 
CAP but thats not in CAP 1.0 and the usefulness is quite limitied.  Again, 
you can simplify this down to the more compact form of:

SELECT * FROM VAGENDA WHERE STATE() = "DELETED"

but so what?  This is not really a common action and the savings in 
storing it is just 47 octets over sending just:

QUERYID:STUFF

>    QUERYID:Everything
>    SELECT VEVENT FROM VEGANDA WHERE STATE() = "UNPROCESSED"
>    SELECT VTODO FROM VEGANDA WHERE STATE() = "UNPROCESSED"
>    SELECT VHOURNAL FROM VEGANDA WHERE STATE() = "UNPROCESSED"
>    SELECT X-MY-COMP FROM VEGANDA WHERE STATE() = "UNPROCESSED"

Nice bloating of the more normalative case:

SELECT * FROM VEGANDA WHERE STATE() = "UNPROCESSED"

which you already covered first.  Nothing new here.

>    QUERYID:MY RIGHTS
>    SELECT VCAR FROM VAGENDA WHERE SELF() IN DENY
>    SELECT VCAR FROM VAGENDA WHERE SELF() IN GRANT

What kind of common action is this?  How often does a CUA search for 
essentially all VCARs where they are either denied or granted some right? 
VCARs managment is NOT a CAP 1.0 concern (its "Administratin" and thus OOB 
as many are keen to point out).

This is not a commonly performed query that I can see.  Hardly a good 
justification to match the original intent.

>    QUERYID:MY CALS
>    SELECT CARID FROM VCALSTORE WHERE SELF() in OWNER

This is another common action?  Finding all calendars in the store that 
the CUA is the owner of?  Again, this is hardly a commonly performed query 
and another poor example that does not match the original intent of stored 
queries.  A CUA doesn't often need to check what calendars they own, they 
should that info already or need to discover it at most once per session.

Thats it?  I will now amend what I said before about no examples; I 
finally got 1.  It saved my CUA sending ~50ish octets.  ~50 octets of what 
can easily be just a hardcoded string that creates no real burden on any 
CUA.  After all, any CUA that can hold a CAP 1.0 engine should not have to 
worry about ~50 more bytes...

Not many examples of any kind to justify such an implicit CS requirement 
to me and only 1 was really a commonly performed queries that matches the 
original intent of the feature.  The savings were trival compared to the 
cost for ALL CS's.  Plus there was NOTHING in any of them that required 
ANY kind of load on the CUA; they could all be resourced strings that just 
get stuffed into the command and sent!

I supported the original intent of stored queries but the current design 
does not approach the intent we created them with.  I still think that 
implicitly requiring ALL CS's to implement this for such little benefit is 
a bad idea. 

I noticed you didnt try to show any queries that did things related to the 
original intent of the stored query.  Nothing about checking for alarm 
triggers.  Nothing about checking for calendar contents for "today" or 
"tomorrow" or "this week" etc.  And why not?  Because these more commonly 
used kinds of queries just cannot be stored and reused without being 
rewritten on some regular basis. 

I have a very strong aversion to implicitly mandating unnecessary features 
in CAP servers, especially when the design does not fulfill the original 
intent.  Why should anyone have to build query storage into their CS when 

A) it does not fulfill the original intent of the feature, 
B) there is nothing in CAP about how a CS can reject stored queries yet 
still say they support VQUERYs that are not stored
C) there is nothing in CAP about the CUAs or CSs responsibility for 
maintenance and cleanup of any stored VQUERY

The single common example provided compared to all the ones we originally 
thought of shows exactly how little benefit there is.  A few octets that 
can be a hardcoded string is hardly compelling reason to keep this feature 
in CAP 1.0.

Why as a CS implementor do I need to design in some query storage 
mechanism because we left in a worthwhile feature that we failed to 
adequately design for in our query language?  I should have the ability to 
reject query storage attempt or any attempt to refer to a non-stored query 
but theres nothing in CAP that allows me to do this nor is there anything 
about how a CUA can or should deal with a CS that tries to reject stored 
queries. 

While I do support the original intent, I think we lost sight of the goal 
and missed the mark on this feature somewhere along the way.  As such lets 
simplify an already complex draft and remove the highly doubious 
requirement/feature.

Bruce
===========================================================================
Bruce Kahn                                INet: 
[email protected]
Messaging & Collaboration                 Phone: 978.399.6496
IBM Software Group                         FAX: and nothing but the FAX...
Standard disclaimers apply, even where prohibited by law...