Re: TMQL - Proposal for a %base directive

Hi Lars,

On 29 Apr 2011, at 9:37 PM, Lars Heuer wrote:

> [...]
>> Unless you're specifying an overlay ontology with your base
>> directive that isn't reflected in the effective map, wouldn't the
>> effective ontology of your query match the effective map by default
>> anyway?  If they didn't have subject indicators, the interpretation
>> of the semantics is context-specific; if they did have subject
>> indicators, then you'd be able to leverage these to ensure fixed
>> semantics because the local identifiers in the effective map would
>> resolve to the same items as the local identifiers in the TMQL query.
> 
> Yes, but what's the base IRI of the effective map? Topic Maps (acc. to
> TMDM) contain always absolute IRIs, they don't have a concept for
> local identifiers. (Topic Maps isn't any different from RDF in this
> regard).

I'd either forgotten this fact or assumed that the preference for absolute IRIs was a remnant of the document-based XTM/LTM/etc. notions of self-contained maps published somewhere on the Web.  I'd always inferred local identifiers within the scope of the map "document" itself would work the same way as I think of them, and things like using '#ident' for local identifiers were syntactic artifacts, not core features.

When you remove the constraint that you're working with physical documents as the container boundary, it's only natural to extend the boundary to an abstract addressable space (via some addressing mechanism) and concentrate on the things you have to work with within that boundary.  Sure, you can maintain references to fully-qualified IRIs, but then - at least to me - these are external to the boundary of what TMQL calls the effective map, so they're a different class of identifiers.

Thus far in my experiments to map to and from my notions and the TMDM serializations, I haven't had trouble.  This may be due to several unarticulated assumptions on my part, however, so thanks for pointing this out.

As for the base IRI of the effective map, isn't this entirely dependent on the access mechanism?

> NB: The fact that Topic Maps (in TMDM) operate with absolute IRIs is
> also one reason why I have my reservations against the "ident isa
> tmql:ontology" mechanism. The "ident" would be an IRI too, and not a
> small, cute local identifier.

I can understand your issue better now.  Thanks.

I'm still of the opinion that these can be solved based on lazy vs. eager identifier resolution.  Depending on your perspective, and when you actually *need* fully-qualified IRIs, I don't think this issue is as serious as you do.

However, as you say, if you ever did need to resolve the ident, yes it would have an IRI that would effectively be an alias for some other construct used in the TMQL query.  Again, though, I don't see this as a real issue.  If you look at the TMQL query and the environment as a map itself, then it too can have a published address, giving everything within it a fully-qualified IRI that would seem to play well with your world view.  I'm not sure how useful this would be in practice, but at least the model scales nicely:  the query-as-map applied as a transformation to the target map to generate another resulting map--exactly along the lines described in TMRMv7.

> TMQL must provide a mechanism to resolve the identifier "foo" into
> some absolute IRI, otherwise queries which use "local" identifiers
> would never return a result. At some point your local identifier must
> become an absolute IRI and the proposed directive makes it more
> explicit which (default) namespace you're using (for item
> identifiers).

I don't agree with the first part about TMQL providing the mechanism based on the above supposition that the query environment itself represents a map applied to the target.  As part of the way that application is performed, either the IRIs would be resolved in terms of the target map (my general assumption and preferred approach that I feel would result in fewest surprises), OR the relative IRIs would be resolved in terms of the query environment itself and require additional help (via subject indicators/locators) to map the TMQL query local identifiers (or their fully-qualified names) to constructs in the target map.

While the latter is certainly possible, I don't think the overhead is either justified or necessary to allow TMQL implementations across both TMRM and TMDM systems to work as people would expect.

I believe that the mechanism you describe is purely a function of the way the particular TMQL query environment works rather than being the responsibility of TMQL to define.  This again is one of the other reasons that I thought a modular TMQL specification would be a good way to go.  If you have the more general and abstract concepts specified in one document, then you can specify particular, standards-compliant mechanisms and behaviors for exposing this functionality based on particular environments like TMDM or in terms of pure TMRM path expressions.  Alternatively, you might just need two, providing the abstract definitions can all be expressed using TMRM path expressions as I suspect they can.

Granted, this is a bit of a cop out because it just moves the problem around, but it at least allows the creation of a standard expected behavior within TMDM environments that doesn't overly constrain other Topic Maps implementations based on other interpretations.

>> Given that most of the rest of you seem to prefer fully-qualified
>> names, that means you're likely to need to use multiple QNames in a
>> given query anyway, so I'm not 100% sure I see the value in this use
>> case.
> 
> Yes, a query would contain many QNames and probably a query wouldn't
> rely on item identifiers but subject identifiers. As said, queries
> like //person are very unlikely since "person" becomes an (absolute)
> item identifier.

I'm not sure I agree with this either.  Most of the use cases I have in mind are interactive queries typed by humans, so fewer characters to type and keeping this as close to what they see in the results makes the most sense to me.  This is a matter of surface presentation rather than the TMQL mechanics, though, so I don't see this as being so critical right now.

> [...]
>> A related question:  what would be the IRIs of the returned items?
>> Aren't they still defined in terms of the effective "base" IRI of
>> the effective map defined by the method of access?
> 
> The IRIs of any information in the topic map(s) wouldn't be altered or
> resolved against the (query) base IRI. Since topic maps contain only
> absolute IRIs they're returned as they are.

That's what I hoped you'd say, but I wanted to make sure.  Thanks.

>>> Given that, I can take my query and go from TM A to TM B without
>>> re-parsing the query.
> 
>> I'm not sure I really understand the implications of this
>> statement.
> 
> Reparsing is probably wrong, I meant, I get a query where the local
> identifiers wouldn't be interpreted differently. Whether I go with my
> query to topic map A or to topic map B, the local identifiers would
> have been resolved already and they don't have to be interpreted
> relative to some topic map.

You could certainly do this (after all, that's what prefixes do), but that's not the same reason I'd want a base directive.  I want to establish a particular perspective as the root so that other relative or system-absolute references are consistently resolved.  I actually have a couple of different levels of addressing in the current implementation, and the value of a base directive for me is mostly under the covers when generating what I would consider "external" views addressable outside the map environment, either via HTTP or some other mechanism.

Still, I think there's enough wiggle room in the way this could be implemented vs. specified so that we could both end up with the behavior we're after from the same specification construct.

> [...]
>>> If it's possible to query multiple maps with TMQL, each identifier
>>> like "tom-waits" has to be resolved relative to each map IRI.
> 
>> This kind of resolution is mostly what I'd expect.  I think the
>> difference is when the expansion actually occurs (at least to me).
> 
> That's correct. I'd prefer that the expansion happens within the query
> parser and that local identifiers become absolute IRIs as early as
> possible and local identifiers wouldn't have to be resolved relative
> to the topic map.

I'm not sure any specification construct should specify more than the intended consequences of its use, so I'd be against saying exactly when any dereferences or expansion happens in the spec.  Putting this sort of stuff in the spec is the kind of thing that prevents optimizations as you observe more about how any environment is actually used, or it optimizes the specification for implementation within a particular environment.  Both of which I think are bad ideas.

Naturally, your TMQL implementation could eagerly expand identifiers because you think that's the right way to go (and it's a perfectly valid perspective), but I still want mine to be able to do lazy resolution as much as is technically possible.

> [...]
>> Second, I think the example above actually illustrates the issues
>> with prefixes vs. proper QNames we've discussed before.  The fact
>> that you're really doing string-concatenation
> 
> No, I don't do simply string-concatenation. IRIs are resolved acc. to
> RFC 3986.

Sorry.  The "you" wasn't supposed to mean "Lars H." in this case.  Generally, the implications of the prefix construct's observable behavior is that it is simply a string concatenation shortcut.  What anyone subsequently does with the resulting string wasn't what I was discussing.

> [...]
>>        [49]    prefix-directive        ::=      %prefix   identifier QIRI
> 
>> This is one of the definitions that's admittedly made me scratch my
>> head a bit because I would've expected an IRI here instead of QIRI.
> 
> Me too. Probably this is simply a mistake in the grammar.

I'd hope so.  It was one of those "implementor interpretations" I was prepared to make myself. :)

>> In my own experience, the only reason to use any kind of base
>> definition is to reduce the typing burden or to apply additional
>> context to something more generally defined.  Either way, it's still
>> a mechanism to help the humans, so, as such, I just didn't see why
>> you wouldn't let people use it as they saw fit.  I wouldn't really
>> see how it would make much difference to the parser.
> 
> Multiple base IRI directives would be possible, I just wanted to keep
> it simple. I think it does more harm than good if foo is sometimes
> (within the same query!) resolved against IRI A and sometimes against
> IRI B. If we allow multiple base IRIs, we can also ask us why prefix
> mappings are immutable and we don't allow multiple bindings to the
> same identifier.

Assumptions of linear processing of query specifications would be my answer to this.  Base directives would be used to resolve/expand unqualified identifiers until those identifiers were used in any context where such use would render them immutable according to the processing requirements defined by the TMQL draft.  Until the identifiers are actually *needed* by the query machinery, they're just shortcuts for typing.  That's why I said that I didn't see an issue with the base directives.

The other thing I had in mind was an implicit "environment/transaction" scope starting from the beginning of the interaction with the TMQL processing environment and the execution of any single query expression.  In this case, I'd agree with you that dynamic reassignment wouldn't be very useful.

However, if you look at an interactive session, similar to psql, mysql or other interactive command line query tools, you'd introduce some kind of query delimiter to denote the boundaries for interpretation.  In this case, you could have a mutable overall environment that would essentially be "frozen" for the execution of each individual TMQL query, so the processing models, bindings, etc. as spec'd would behave appropriately.

I guess I was mixing usage patterns with my emails, so apologies for that.  How a multi-query, interactive session would actually behave is well beyond the boundaries and scope of the TMQL specification.

> [...]
>> If tolog actually pre-resolves the local identifiers relative to
>> the local map as part of the parsing phase, I'd view that as an over-eager optimization.
> 
> I think that's not specified. The tolog specification simply says that
> identifiers become item identifers and are interpreted relative to the
> current map's IRI.

Makes sense to me, and I wouldn't have necessarily expected any more than that.

Cheers,

ast
--
Andrew S. Townley <[email protected]>
http://atownley.org