Re: Afghanistan War Diary as topic map in Maiana

Aki,

Thanks for the work on the map and the report on the extractors!

Do any of the extraction services add properties to what we would call
subjects? 

The entity extraction stuff is impressive but only a step in the right
direction.

Imagine that you have entity extraction that identifies a judicial
appointee, the court, supporters, opponents, etc. OK, but having an
information appliance that recognizes that is the appointee is the same
person made substantial donations to the person appointing them, that is
semantic integration of a different color. 

Hope you are at the start of a great week!

Patrick

On Mon, 2010-11-01 at 10:00 +0200, Aki Kivela wrote:
> 
> Hello All
> 
> Ongoing discussion about the quality and usability of the Afganistan War 
> Diary topic map inspired me trying to enhance the topic map using 
> Wandora's extractors. Most information in the AWD topic map is stored in 
> occurrences i.e. plain unstructured text. Wandora has several 
> information extractors that distill topics and associations out of 
> unstructured text. These extractors use external web service apis 
> provided by Calais [1], AlchemyAPI [2] and Yahoo [3]. I choosed to 
> experiment with following extractors
> 
>   * Alchemy Entity extractor [4]
>   * Alchemy Keywords extractor [4]
>   * OpenCalais extractor [5]
>   * Yahoo! term extractor [6]
> 
> I applied each extractor, one by one, to all summary occurrences of 
> report topics. Number of report topics was 1987. Summary occurrence 
> contains textual representaion of the military event the report talks 
> about. Applying an extractor to occurrences generated more topics and 
> associations. Outcome of each extractor was stored to a separate topic map.
> 
> I have packed all source and extracted topic maps to a zip package at 
> [7]. I also tried to upload everything to Maiana but had some technical 
> difficulties. In practice you should be able to merge any of the 
> generated topic maps with the original AWD and you should see extracted 
> topics associated with the original report topic. As the original 
> summary occurrence is available, anyone can evaluate if the extracted 
> topics and associations really describe the report.
> 
> To summarize results. Yahoo term extractor found 3563 term topics and 
> 15092 associations in occurrences. OpenCalais found 18 distict topic 
> classes (topic doesn't refer here to a topic concept of Topic Maps) and 
> tags in 25 categories. OpenCalais extractor generated 3455+1317 
> associations. Alchemy entity extractor found 424 entities (1285 
> associations), and Alchemy keyword extractor 9054 keywords (14684 
> associations) in occurrence texts.
> 
> Although the technical implementation of the experiment was easy and 
> went fine, I am not really sure about the quality of automatic 
> classifications provided by Calais, Alchemy and Yahoo. It is clear that 
> the source material, reports of military actions, is very challenging 
> due to military specific expressions, terms, and acronyms, and it looks 
> like all classifiers have made false interpretations. It would be very 
> interesting if someone would like to evaluate the overall quality of 
> generated topic maps and point out typical error classes.
> 
> In any case, I hope this demonstration clearly shows a simple topic map 
> storing merely occurrence data is not necessarily a dead end but a good 
> start.
> 
> Kind Regards,
> Aki Kivelä
> Wandora Team
> 
> [1] http://www.opencalais.com/
> [2] http://www.alchemyapi.com/
> [3] http://developer.yahoo.com/search/content/V1/termExtraction.html
> [4] 
> http://www.wandora.org/wandora/wiki/index.php?title=AlchemyAPI_extractors
> [5] 
> http://www.wandora.org/wandora/wiki/index.php?title=OpenCalais_classifier
> [6] This extractor is not part of official Wandora version yet.
> [7] http://www.wandora.org/wandora/download/other/AWD2004_experiment.zip
> _______________________________________________
> topicmapmail mailing list
> [email protected]
> http://www.infoloom.com/mailman/listinfo/topicmapmail
_______________________________________________
topicmapmail mailing list
[email protected]
http://www.infoloom.com/mailman/listinfo/topicmapmail