Re: gedcom diff

I have been looking into "Probabilistic Record Linkage and 
Deduplication", you might want to throw that into google without the 
quotes. These are methods the allow you to set up weights if fields 
exist or don't exist and compare records. You get an overall score for 
each record comparison in terms of 0.0 .. 1.0 or the %age likelihood of 
a match. These are methods used in databases to match records from 
various sources that may have missing, incomplete, or misspelled, or 
other "noise". Some of these techniques use HMM (Hidden Markoff Models).

So I have been looking into these for Geocoding and Address 
Standardization, but it could also be applied to Gedcom record 
(individual) matching. This is not trivial stuff to understand but from 
what I have read could be a VERY powerful tool for doing this kind of 
matching in the genealogy world. I have too many projects going at the 
moment to work on this, but I would be happy to point anyone that is 
interested in the right direction and bounce ideas around the list on it.

Does anyone else know anything about these ideas?

-Steve W.

Hugh S. Myers wrote:
> I think the idea of root based tree comparison is a good one and a diff
> style output likewise excellent. That said I would also implement a 'fuzzy'
> comparison with particular attention paid to name similarity. Perhaps
> rolling in some sort of rated value based on the sound of the names as well
> (name of this kind comparison slipped my mind as soon as I wrote this
> down...)
> 
> --hsm
> 
> 
>>-----Original Message-----
>>From: Hans Fugal [mailto:[email protected]]
>>Sent: Monday, March 13, 2006 6:53 AM
>>To: [email protected]
>>Subject: Re: gedcom diff
>>
>>Sorry for the quite late reply - you can see how often I wade through
>>secondary lists when I get busy. :)
>>
>>I've thought about a more literal gedcom diff - you give it two gedcoms
>>and a root pair, and it spits out exactly what the differences are in
>>some easy-to-digest way. I was thinking regular diff-like output,
>>actually, which could be interpreted by gedcom-savvy humans or by a
>>program with a UI. The hard part was deciding which sort order to put
>>the records in because the greatly influences how the diff looks.
>>
>>I remember thinking along the same lines as you about comparing trees,
>>and I think that's the right approach, especially given a known root
>>pair. I think you would still want to compare names or some other
>>identifying characteristics in addition to just tree shape, which would
>>prevent merging multiple parents/spouses, not to mention children!
>>
>>On Tue, 16 Nov 2004 at 06:49 -0700, [email protected] wrote:
>>
>>>Has anyone come up with code that can diff two .ged files? I have two
>>>large files with data missing from each (two sides of the family). All
>>
>>the
>>
>>>tools I have looked at seem to use soundex to find matches and my test
>>>merge has not gone well (I just used the paf5 match/merge).
>>>
>>>I was thinking if I wrote code where I would give it a reference person
>>>from each file "these two are the same", the code could build a tree
>>
>>from
>>
>>>there and then compare the trees not the names themselves to tell me
>>
>>which
>>
>>>nodes were missing. I am still not sure how I would handle multiple
>>>spouses, and multiple parents and such.... but I have been mulling it
>>>around for a while.
>>>
>>>Thought I would see if anyone had built anything along these lines.
>>>
>>>--
>>>Chuck
>>>
>>
>>--
>>Hans Fugal ; http://hans.fugal.net
>>
>>There's nothing remarkable about it. All one has to do is hit the
>>right keys at the right time and the instrument plays itself.
>>    -- Johann Sebastian Bach
> 
> 
>