Re: Semantic Interpretation of Block Structure

   From: Chris Babcock <[email protected]>

   [...]

   I'm looking for tokenization strategies to appropriately weight semantic
   mark up, specifically whether anyone has already experimented with the
   effect of using tokens for paragraph and other block elements.

Yes.   In fact, one of the recent MIT Spam Conference papers did
exactly that- _ignored_ the actual text and replaced it with
signatures of the markup.

Things like long URLs, short URLs, single-line paragraphs, long
paragraphs, etc got markers.  Things like the the time and date get
translated into a format string, so it didn't matter what time or
date something was sent, but how that string was encoded _did_ matter
(i.e. 10 April is different than April 10 or 10-April).

Notice that all of the textural content is stripped away by this
algorithm.  Nothing remains but formatting information....

... and it worked pretty well!

    - Bill Yerazunis

------------------------------------------------------------------------------
Stay on top of everything new and different, both inside and 
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today. 
Use priority code J9JMT32. http://p.sf.net/sfu/p