Re: Semantic Bleachers: was Re: Afghanistan War Diary as topic map in Maiana

On 1 Nov 2010, at 6:55 PM, Patrick Durusau wrote:

> Andrew,
> 
> On Mon, 2010-11-01 at 06:32 +0000, Andrew S. Townley wrote:
>> Hi Patrick,
>> 
>> On 1 Nov 2010, at 1:11 AM, Patrick Durusau wrote:
>>> 
>>> PS: I think assigning properties to subjects and then saying on what
>>> basis we think they are the same as other subjects avoids the pitfalls
>>> of the OWL sameAs which has no properties nor any way for a user to
>>> indicate on what basis they claimed "sameAs." 
>> 
>> 
>> This point is so absolutely critical so that people understand the basis on which an equality assertion is being made.  I've long held that equality is nothing more than having a particular set of properties the same, even if those are a subset of properties and the equality is only intended to exist in a limited context.  It's on the basis of property values, not magic ids, that define whether or not a given proxy (or topic) should be claimed to the same subject in a given context.
>> 
> 
> Actually whether the properties and their values are the "same," is a
> decision I would leave up to the legend designer. 
> 
> That is there could be a rule that says:
> 
> If fullname = "Patrick Durusau" and SSN = "***-**-****", same subject,
> so that merging would happen even if those properties were in separate
> containers. 
> 

I think I agree, but what do you mean by "container" here?  Is it a property?  Is it a proxy?  Is it an alias for a property across proxy instances?

Basically, this type of merging is what I've done in my implementation so far.  It also does a "same type" & "same property" & "same value" check that will determine whether merging can happen or not.

> I think the creative fashioning of merging rules, with boundaries for
> their application is largely unexplored territory. 
> 
> We would benefit from a very robust set of comparison operators. 

That's what I was thinking too, but I wasn't sure.  Again, I don't see how you can deal with Internet-scale datasets if you don't have these type of "overlay" facilities that can be linked to specific contexts.  Simple ones are "I'm user X" and "She's user Y", but others could be a lot more sophisticated.

However, from personal experience, this stuff gets pretty complicated (and CPU intensive) pretty quickly.  Still working on ways to deal with that issue to my satisfaction...

> 
>> I think you alluded to this the other day with some of your work, but I think that the idea behind what Manina is doing, what I understood was possible with what you have and the way that my implementation handles merging on a conditional rather than absolute basis illustrates that there's still work to be done in this area of the topic maps specifications.  Maybe some implementations take too literal a view on the whole XTM/TMDM merging requirements, but it makes a big deal if you define "equality" on the basis of property values, merge and then one or more of those property values change.
>> 
> 
> Well, those are separate questions:
> 
> a. How to merge on something in addition to the standard TMDM basis?

Or, in our case, "in spite of" or "happily ignoring" the TMDM basis... ;)

> 
> b. What happens if after merging a value changes? (unexplored so far as
> I know, but good question)

The way I see it, "merging" by definition is just another context of viewing a particular dataset.  What's available and the legend will define the outcome.  This outcome is totally linked to the context, although the validity of the context can be "until further notice" or "from timestamp T where properties [...] also apply."

Complexity explosion?  Absolutely, but, again, I think it's essential to work through these scenarios.  "Truth" is relative, and the results of any view of any data set on any given day represent one particular view of "truth" in that domain.  Expand the scope and scale, and the differences between "truth" start to be interesting as artifacts of themselves, but without the traceability and the ability to do "what if" scenario modeling, this can be very difficult to determine.

Would really be interested in understanding what large TMDM users with real data do in these sorts of scenarios, or if they're just ignored.  Has anyone done any kind of analysis of the way the specs are applied and the corresponding trade-offs that result from going down these paths?

I'm not a fan of "best practices" because that tends to end up being a fluffy middle-ground of ambiguity that keeps people able to report upwards that "we're following best practices" when results are questioned.  What I think we need are more detailed analysis of the issues people are facing trying to employ topic maps (of any flavor) to solve interesting (and/or business) problems beyond single examples published at topic maps conferences.  Has this been done?  If so (and I hope), I'd love to see it, because I think it would be fascinating reading.

> 
>> How do other implementations handle this scenario?  To me the answer is key to dealing with large, somewhat ambiguous data sets like the Afghan one that stated the discussion and allowing the humans using them to derive real value from them--regardless what encoding and representation they start or finish in.
>> 
> 
> I don't know.
> 
> My personal interest is in functional topic maps that display "merged"
> information as it is requested.
> 
> In that case, the answer to your question of what happens if a value
> changes after merging is likely to be different from file oriented
> systems. 
> 

Interesting.  I think we're after similar goals here.  What exactly do you mean by "functional" topic  maps in this case?

--
Andrew S. Townley <[email protected]>
http://atownley.org