Re: Afghanistan War Diary as topic map in Maiana

Hello All

Ongoing discussion about the quality and usability of the Afganistan War 
Diary topic map inspired me trying to enhance the topic map using 
Wandora's extractors. Most information in the AWD topic map is stored in 
occurrences i.e. plain unstructured text. Wandora has several 
information extractors that distill topics and associations out of 
unstructured text. These extractors use external web service apis 
provided by Calais [1], AlchemyAPI [2] and Yahoo [3]. I choosed to 
experiment with following extractors

  * Alchemy Entity extractor [4]
  * Alchemy Keywords extractor [4]
  * OpenCalais extractor [5]
  * Yahoo! term extractor [6]

I applied each extractor, one by one, to all summary occurrences of 
report topics. Number of report topics was 1987. Summary occurrence 
contains textual representaion of the military event the report talks 
about. Applying an extractor to occurrences generated more topics and 
associations. Outcome of each extractor was stored to a separate topic map.

I have packed all source and extracted topic maps to a zip package at 
[7]. I also tried to upload everything to Maiana but had some technical 
difficulties. In practice you should be able to merge any of the 
generated topic maps with the original AWD and you should see extracted 
topics associated with the original report topic. As the original 
summary occurrence is available, anyone can evaluate if the extracted 
topics and associations really describe the report.

To summarize results. Yahoo term extractor found 3563 term topics and 
15092 associations in occurrences. OpenCalais found 18 distict topic 
classes (topic doesn't refer here to a topic concept of Topic Maps) and 
tags in 25 categories. OpenCalais extractor generated 3455+1317 
associations. Alchemy entity extractor found 424 entities (1285 
associations), and Alchemy keyword extractor 9054 keywords (14684 
associations) in occurrence texts.

Although the technical implementation of the experiment was easy and 
went fine, I am not really sure about the quality of automatic 
classifications provided by Calais, Alchemy and Yahoo. It is clear that 
the source material, reports of military actions, is very challenging 
due to military specific expressions, terms, and acronyms, and it looks 
like all classifiers have made false interpretations. It would be very 
interesting if someone would like to evaluate the overall quality of 
generated topic maps and point out typical error classes.

In any case, I hope this demonstration clearly shows a simple topic map 
storing merely occurrence data is not necessarily a dead end but a good 
start.

Kind Regards,
Aki Kivelä
Wandora Team

[1] http://www.opencalais.com/
[2] http://www.alchemyapi.com/
[3] http://developer.yahoo.com/search/content/V1/termExtraction.html
[4] 
http://www.wandora.org/wandora/wiki/index.php?title=AlchemyAPI_extractors
[5] 
http://www.wandora.org/wandora/wiki/index.php?title=OpenCalais_classifier
[6] This extractor is not part of official Wandora version yet.
[7] http://www.wandora.org/wandora/download/other/AWD2004_experiment.zip