Re: The AF war diary

Patrick, Lars,

First, I wanted to comment on Patrick's Blog post, but it doesn't
allow comments for posts older than 2 days.

On Fri, Jan 28, 2011 at 5:21 PM, Lars Heuer <[email protected]> wrote:
> Hi Patrick,
>
> [...]
>> Aki has a good point that occurrence data in such an topic map SQL dump
>> can be the *starting point for further refinement.* And he actually
>> posted refinement data to Maiana.
>
> Yes, I think that's the right direction.

I also think that's the right direction - without even having a closer
look at it. Let me bring some light on how it all started. One day
somebody¹ came into my office (which I share with Thomas) and
disrupted us from our work, like it happens every day.
He talked about this hot Wikileaks thing everybody is writing about
and what effort it would be to get it into a topic map, i.e. not do
anything to it except making it available somehow. Our answer was
short: close to none: Download the CSV, run some regular expressions
on it, save it as CTM and upload it to Maiana. Done in a few minutes
by a student assistant.
While gazing at the horrible result, he found that there are many
typos, even in the category names, so he decided to load the CSV into
Freebase Gridworks (which is called Google Refine nowadays) and export
it from there to CTM which was done in a few minutes, too².
He uploaded it to Maiana again, wrote a short note to the "somebody"
mentioned above and then he continued reading papers for his thesis.

¹ you may guess who it was but not publish it :P
² If I remember correctly, the only things which thwarted him for a
second was strange literal escaping and the use (and even inconsistent
use) of literals in CTM.

>> Sure, I think the TopicMaps Lab should be the origin of such refinement
>> but apparently they have other priorities.

Indeed they have.

> Apparently some see a value in the topic map as it is otherwise they
> wouldn't peddle around with it.

Indeed somebody sees a value in it. People are even writing mails and
ask us to present it on conferences. The main point of the talk we're
going to give is not how great the result is but the little effort
which was needed to obtain it. I guess Aki and Lars each spent 10
times more time on thinking about it than Thomas and I spent on it
together. Now where are your 10 times better results, Lars? I can see
Aki's.

> That point leads me to my initial question: How could Der Spiegel
> could have benefited from the topic map? I'd be glad it someone of the
> creators would stand up for the thesis they brought up.

Probably not so much with this half-an-hour-exercise conversion. But
the above-mentioned "somebody" had the idea of putting additional data
into the same topic map: The major cities and their geo coordinates.
This completely separate additional data didn't even trigger any
merging, but it provided a tiny bit of added value nevertheless:

%prefix wds http://psi.topicmapslab.de/wardiary/schema/
for $r in fn:coordinates-in-distance( "Kabul" << atomify tm:name <<
characteristics / wds:coordinates , 30 ) return $r << characteristics
/ wds:hostnationwia

One could e.g. query for host nation WIA incidents near Kabul directly
without having to look in one place to get the coordinates of Kabul
and then put them into a query. Instead of looking for things which
have the name Kabul and getting it's coordinates, one could also just
use e.g. the geonames identifier. Compared to simple text mining, this
would also incidents where Kabul is not mentioned in the description.

Of course this could be done with many other technologies, too. The
benefit compared to CSV and an SQL dump is just having all your
additional knowledge in one place and being able to combine it.
Or is there anything on the web where a non-technical person can
upload SQL dumps and/or CSV dumps and do structured queries on it?

By the way: With these few steps of adding identifiers to CSV cells we
improved queryability and therfore made the data available to the LOD
who can use the Sparql end point. What are your 3 steps in less than
30 minutes to convert the raw data into linked open data?

What could Spiegel have done? I guess they may have a map of census
data for larger cities there. Or maybe the locations of whereabouts of
their reporters. Just throw this data into the topic map and run some
useful queries. Build hypotheses based on the statistics, verify it
according to matching incidents or see that it doesn't hold against
the reports.

It took a while to write down this E-Mail, so maybe I'm now on par
with Lars' efforts on (working on UNION discussing) the AWD map. Now
what about your results, Lars?

One remark regarding Lars' and Robert's remarks on how to not model a
topic map: We all know and agree that a 1:1 mapping of relational data
is not what one may considers a beautiful topic map, but who except
you cares? It's the same question as if the TMDM is beautiful or not.
Most people (except maybe the readers of this mailing list) do not
have their mental models in terms of a topic map. People like lists of
data and tables of data. That's one reason they use these structures
all the time. (The other is: they are used to the tools which provide
easy working with these structures.)
Just transforming easy-to-grasp table-like data into a graph does not
provide much value either (or at least it wouldn't be worth the effort
needed with currently available tools). Topic Maps come handy when you
want to describe the existing structure (i.e. the 1:1 mapping) and
match it with other existing structures (i.e. other 1:1 mappings). I
think nobody want's to spend hours of time to transform data to
another structure in the hope of maybe better results. It also cannot
easily be done automatically. I'm not a reporter and not really
interested in the incidents themselves, so I won't spend a lot of time
in unearthing subliminal data. In my point of view, it was just a
quick exercise to see how fast it can be done and how the tools
available perform.

Best regards,
Benjamin
_______________________________________________
topicmapmail mailing list
[email protected]
http://www.infoloom.com/mailman/listinfo/topicmapmail