Added: lenya/branches/BRANCH_2_0_X/src/modules/editors/resources/codemirror/0.61/manual.html
URL: http://svn.apache.org/viewvc/lenya/branches/BRANCH_2_0_X/src/modules/editors/resources/codemirror/0.61/manual.html?rev=794002&view=auto
==============================================================================
--- lenya/branches/BRANCH_2_0_X/src/modules/editors/resources/codemirror/0.61/manual.html (added)
+++ lenya/branches/BRANCH_2_0_X/src/modules/editors/resources/codemirror/0.61/manual.html Tue Jul 14 18:14:46 2009
@@ -0,0 +1,575 @@
+<html xmlns="http://www.w3.org/1999/xhtml">
+ <head>
+ <title>CodeMirror user manual</title>
+ <link rel="stylesheet" type="text/css" href="css/docs.css"/>
+ </head>
+ <body>
+ <h1 class="underline">CodeMirror user manual</h1>
+
+ <h2>Contents</h2>
+
+ <ul>
+ <li><a href="#useage">Basic Useage</a></li>
+ <li><a href="#configuration">Configuration</a></li>
+ <li><a href="#parsers">Parsers</a></li>
+ <li><a href="#programming">Programming Interface</a></li>
+ <li><a href="#writeparser">Writing a Parser</a></li>
+ </ul>
+
+ <h2 id="useage">Basic Usage</h2>
+
+ <p>Inside the editor, the tab key is used to re-indent the current
+ selection (or the current line when nothing is selected), and
+ pressing enter will, apart from inserting a line break,
+ automatically indent the new line. Pressing control-enter will
+ cause the whole buffer to be re-coloured, which can be helpful
+ when some colouring has become out-of-date without the editor
+ noticing it.</p>
+
+ <p>The editor sports an undo/redo system, accessible with
+ control-z (undo) and control-y (redo). Safari will not allow
+ client scripts to capture control-z presses, but you can use
+ control-backspace instead on that browser.</p>
+
+ <p>The key-combination control-[ triggers a paren-blink: If the
+ cursor is directly after a '(', ')', '[', ']', '{', or '}', the
+ editor looks for the matching character, and highlights these
+ characters for a moment. There is an option to enable this to
+ happen any time the user types something or moves the cursor.</p>
+
+ <p>To use CodeMirror in a document, you should add a script tag to
+ load <a href="js/codemirror.js"><code>codemirror.js</code></a>. This
+ adds two objects to your environment, <code>CodeMirror</code> and
+ <code>CodeMirrorConfig</code>. The first is the interface to the
+ editor, the second can be used to configure it. (Note that this is
+ the only name-space pollution you can expect from CodeMirror --
+ all other cruft is kept inside the IFRAMEs that it creates when
+ you open an editor.)</p>
+
+ <p>To add an editor to a document, you must choose a place, a
+ parser, and a style-sheet for it. For example, to append an
+ XML editor to the body of the document, you do:</p>
+
+ <pre class="code">var editor = new CodeMirror(document.body, {
+ parserfile: "parsexml.js",
+ stylesheet: "xmlcolors.css"
+});</pre>
+
+ <p>The first argument to the <code>CodeMirror</code> constructor
+ can be a DOM node, in which case the editor gets appended to that
+ node, or a function, which will be called with the IFRAME node as
+ argument, and which is expected to place that node somewhere in
+ the document.</p>
+
+ <p>The second (optional) argument is an object that specifies
+ options. A set of default options (see below) is present in the
+ <code>CodeMirrorConfig</code> object, but each instance of the
+ editor can be given a set of specific options to override these
+ defaults. In this case, we specified that the parser should be
+ loaded from the <a
+ href="js/parsexml.js"><code>"parsexml.js"</code></a> file, and
+ that <a href="css/xmlcolors.css"><code>"xmlcolors.css"</code></a>
+ should be used to specify the colours of the code.</p>
+
+ <p>Another example:</p>
+
+ <pre class="code">var editor = new CodeMirror(CodeMirror.replace("inputfield"), {
+ parserfile: ["tokenizejavascript.js", "parsejavascript.js"],
+ path: "lib/codemirror/js/",
+ stylesheet: "lib/codemirror/css/jscolors.css",
+ content: document.getElementById("inputfield").value
+});</pre>
+
+ <p>Here we use the utility function
+ <code>CodeMirror.replace</code> to create a function that will
+ replace a node in the current document (given either directly or
+ by ID) with the editor. We also select the JavaScript parser this
+ time, and give a <code>path</code> option to tell the editor that
+ its files are not located in the same directory as the current
+ HTML page, but in <code>"lib/codemirror/"</code>.</p>
+
+ <p>There is a function
+ <code>CodeMirror.isProbablySupported()</code> that causes some
+ 1998-style browser detection to happen, returning
+ <code>false</code> if CodeMirror is probably not supported on the
+ browser, <code>true</code> if it probably is, and
+ <code>null</code> if it has no idea. As the name suggests, this is
+ not something you can rely on, but it's usually better than
+ nothing.</p>
+
+ <p>Another utility function, <code>CodeMirror.fromTextArea</code>,
+ will, given a textarea node or the id of such a node, hide the
+ textarea and replace it with a CodeMirror frame. If the textarea
+ was part of a form, an <code>onsubmit</code> handler will be
+ registered with this form, which will load the content of the
+ editor into the textarea, so that it can be submitted as normal.
+ This function optionally takes a configuration object as second
+ argument.</p>
+
+ <pre class="code">var editor = CodeMirror.fromTextArea("inputfield", {
+ parserfile: ["tokenizejavascript.js", "parsejavascript.js"],
+ path: "lib/codemirror/js/",
+ stylesheet: "lib/codemirror/css/jscolors.css"
+});</pre>
+
+ <p>The reason that the script path has to be configured is that
+ CodeMirror will load in a bunch of extra files when an editor is
+ created (the parser script, among others). To be able to do this,
+ it has to know where to find them. These are all the JavaScript
+ files that are part of CodeMirror itself:</p>
+
+ <dl>
+ <dt><a href="js/codemirror.js"><code>codemirror.js</code></a></dt>
+ <dd>Main interface, takes care of default configuration and the
+ definition of editor frames. Include this in your HTML
+ document.</dd>
+ <dt><a href="js/editor.js"><code>editor.js</code></a></dt> <dd>The
+ code that takes care of reacting to user input, colouring text,
+ and indenting lines.</dd>
+ <dt><a href="js/util.js"><code>util.js</code></a></dt> <dd>A few
+ generic utility functions.</dd>
+ <dt><a
+ href="js/undo.js"><code>undo.js</code></a></dt>
+ <dd>Implements the undo history for the editor.</dd>
+ <dt><a
+ href="js/stringstream.js"><code>stringstream.js</code></a></dt>
+ <dd>Objects for wrapping the textual input to the parser.</dd>
+ <dt><a href="js/select.js"><code>select.js</code></a></dt> <dd>Some
+ helper utilities for working with selected text and cursor
+ positions.</dd>
+ <dt><a href="js/tokenize.js"><code>tokenize.js</code></a></dt>
+ <dd>Helper framework for writing tokenisers.</dd>
+ </dl>
+
+ <p>Most of these are rather full of comments, which can be useful
+ when you are trying to figure out how they work, but wastes a lot
+ of bandwidth in a production system. Take a look at the
+ description of the <code>basefiles</code> option below if you want
+ to concatenate and minimise the library.</p>
+
+ <p>Apart from these, there are files that implement the various
+ parsers. These all start with either <code>parse</code> or
+ <code>tokenize</code>.</p>
+
+ <h2 id="configuration">Configuration</h2>
+
+ <p>There are three ways to configure CodeMirror:</p>
+
+ <ul>
+ <li>If you define a global <code>CodeMirrorConfig</code> object
+ before loading <a
+ href="js/codemirror.js"><code>codemirror.js</code></a>, the
+ configuration options in that object will override the
+ defaults.</li>
+ <li>By assigning to the properties of the
+ <code>CodeMirrorConfig</code> object, configuration defaults can
+ be overridden after loading <a
+ href="js/codemirror.js"><code>codemirror.js</code></a>.</li>
+ <li>The <code>CodeMirror</code> constructor can be given a second
+ argument, an object, which will override some options for just
+ that editor. Options not mentioned in this object will default to
+ the values in the <code>CodeMirrorConfig</code> object.</li>
+ </ul>
+
+ <p>The options that can be specified are these (most have sensible
+ defaults specified in <a
+ href="js/codemirror.js"><code>codemirror.js</code></a>):</p>
+
+ <dl>
+
+ <dt><code>stylesheet</code></dt><dd>The file name of the style-sheet
+ that should be used to colour the code in the editor frame. See <a
+ href="css/jscolors.css"><code>jscolors.css</code></a> for an
+ example.</dd>
+
+ <dt><code>path</code></dt><dd>The path that is prefixed to
+ script file names when they are loaded into an IFRAME. (Note that
+ this is not applied to the style-sheet file name.)</dd>
+
+ <dt><code>parserfile</code></dt><dd>A file name string, or an
+ array of strings that name the files containing the parser. See
+ below for the interface that such a parser should
+ implement.</dd>
+
+ <dt><code>basefiles</code></dt><dd>An array of strings naming
+ the files containing the base CodeMirror functionality. Defaults
+ to <code>["util.js", "stringstream.js", "select.js", "undo.js",
+ "editor.js", "tokenize.js"]</code>, but if you put them all into
+ a single file to reduce latency, or add some functionality, you
+ might have to adjust that.</dd>
+
+ <dt><code>linesPerPass</code></dt><dd>Specifies the maximum
+ amount of lines that the highlighter tries to colour in one
+ shot. Setting this too high will cause the code to 'freeze' the
+ browser for noticeable intervals. Defaults to 30.</dd>
+
+ <dt><code>passDelay</code></dt><dd>Gives the amount of
+ milliseconds between colouring passes. Defaults to 200.</dd>
+
+ <dt><code>continuousScanning</code></dt><dd>Configure continuous
+ scanning of the document. When <code>false</code>, scanning is
+ disabled. When set to a number, say <code>N</code>, a
+ 'background' process will scan <code>linesPerPass</code> (see
+ above) lines of the document every <code>N</code> milliseconds,
+ regardless of whether anything changed. This makes sure
+ non-local changes propagate through the document, and will help
+ keep everything consistent. It does add extra processing cost,
+ even for an idle editor. Default value is
+ <code>false</code>.</dd>
+
+ <dt><code>autoMatchParens</code></dt><dd>When <code>true</code>,
+ will cause parens to be matched every time a key is pressed or
+ the user clicks on the document. Defaults to <code>false</code>.
+ Might be expensive for big documents.</dd>
+
+ <dt><code>saveFunction</code></dt><dd>If given a function
+ value, that function will be invoked when the user presses
+ control-s. You should advise your Opera users to use
+ control-shift-s instead, since plain control-s will bring up the
+ 'save page' dialog. Defaults to <code>null</code>.</dd>
+
+ <dt><code>undoDepth</code></dt><dd>Maximum length of the undo
+ history. Default is 50.</dd>
+
+ <dt><code>onChange</code></dt><dd>An optional function of zero
+ arguments that gets called whenever the document is changed.
+ Happens at undo-commit time, not instantaniously.</dd>
+
+ <dt><code>undoDelay</code></dt><dd>When nothing is done in the
+ editor for this amount of milliseconds, pending changes get
+ added to the undo history. Setting this lower will give the undo
+ functionality a finer granularity. Defaults to 800.</dd>
+
+ <dt><code>width</code>, <code>height</code></dt><dd>The size of
+ the editor frame, given as a style-sheet quantities (for example
+ <code>"600px"</code> or <code>"100%"</code>).</dd>
+
+ <dt><code>disableSpellcheck</code></dt><dd>Should the editor
+ disable spell-checking on browsers that support it (Firefox 2+).
+ Default is <code>true</code>, since for most code spell-checking
+ is useless.</dd>
+
+ <dt><code>textWrapping</code></dt><dd>Can be used to disable or
+ enable text-wrapping in the editor frame. Default is
+ <code>true</code>.</dd>
+
+ <dt><code>lineNumbers</code></dt><dd>Show line numbers to the
+ left of the editor. This requires you to specify a style for the
+ <code>CodeMirror-line-numbers</code> CSS class (in the outer
+ document) to configure the width, font, colors, etcetera for the
+ line-number DIV. You have to make sure that lines in the
+ numbering element have the same height as lines in the editor.
+ This is most easily done by giving them both the same font and
+ an absolute ('pt' or 'px') font size. This option defaults to
+ <code>false</code>. When enabling this, you have to disable
+ <code>textWrapping</code>, since the line numbers don't take
+ wrapped lines into account.</dd>
+
+ <dt><code>indentUnit</code></dt><dd>An integer that specifies
+ the amount of spaces one 'level' of indentation should add.
+ Default is <code>2</code>.</dd>
+
+ <dt><code>tabMode</code></dt><dd>Determines what the effect of
+ pressing tab is. Possibilities are:
+ <dl>
+ <dt><code>"indent"</code></dt><dd>The default. Causes tab to
+ adjust the indentation of the selection or current line using
+ the parser's rules.</dd>
+ <dt><code>"spaces"</code></dt><dd>Pressing tab simply inserts
+ four spaces.</dd>
+ <dt><code>"default"</code></dt><dd>CodeMirror does not
+ interfere with the tab key, but leaves it to the browser to
+ handle it. Binds shift-space to regular indentation
+ behaviour.</dd>
+ <dt><code>"shift"</code></dt><dd>Pressing tab indents the
+ current line (or selection) one <code>indentUnit</code>
+ deeper, pressing shift-tab or ctrl-tab (whichever your browser
+ does not interfere with), un-indents it.</dd>
+ </dl></dd>
+
+ <dt><code>readOnly</code></dt><dd>When set to <code>true</code>,
+ the document is not editable.</dd>
+
+ <dt><code>initCallback</code></dt><dd>If set to a function, this
+ will be called (with the editor object as its argument) after
+ the editor has finished initialising.</dd>
+
+ <dt><code>cursorActivity</code></dt><dd>A function that is
+ called every time the cursor moves, with the top-level node that
+ the cursor is inside or next to as an argument. Can be used to
+ have some controls react to the context of the cursor.</dd>
+
+ <dt><code>activeTokens</code></dt><dd>Can be set to a function
+ that will be called with <code>(spanNode, tokenObject,
+ editor)</code> arguments whenever a new token node is being
+ added to the document. Can be used to do things like add event
+ handlers to nodes. Should <em>not</em> change the DOM structure
+ of the node (so no turning the span into a link), since this
+ will greatly confuse the editor.</dd>
+
+ <dt id="parserConfig"><code>parserConfig</code></dt><dd>An
+ object value that is passed along to the parser to configure it.
+ What this object should look like depends on the parser
+ used.</dd>
+
+ <dt><code>content</code></dt><dd>The starting content of the
+ editor. You'll probably not want to provide a global default for
+ this, but add it to the <code>options</code> object passed to
+ individual editors as they are created.</dd>
+
+ </dl>
+
+ <h2 id="parsers">Parsers</h2>
+
+ <p>The following parsers come with the distribution of CodeMirror:</p>
+
+ <dl>
+ <dt><code><a
+ href="js/parsexml.js">parsexml.js</a></code></dt><dd>A HTML/XML
+ parser. Takes a <code>useHTMLKludges</code> configuration option
+ (see the <code><a href="#parserConfig">parserConfig</a></code>
+ option above), which specifies whether the content of the editor
+ is HTML or XML, and things like self-closing tags
+ (<code>br</code>, <code>img</code>) exist. This defaults to
+ <code>true</code>. Example colours for the styles that this
+ parser uses are defined in <code><a
+ href="css/xmlcolors.css">css/xmlcolors.css</a></code>.</dd>
+
+ <dt><code><a
+ href="js/tokenizejavascript.js">tokenizejavascript.js</a></code>,
+ <code><a
+ href="js/parsejavascript.js">parseejavascript.js</a></code></dt><dd>The
+ JavaScript parser. Example colours in <code><a
+ href="css/jscolors.css">css/jscolors.css</a></code></dd>
+
+ <dt><code><a
+ href="js/parsecss.js">parsecss.js</a></code></dt><dd>A CSS
+ parser. Styles in <code><a
+ href="css/csscolors.css">css/csscolors.css</a></code></dd>
+
+ <dt><code><a
+ href="js/parsehtmlmixed.js">parsehtmlmixed.js</a></code></dt><dd>A
+ mixed-mode HTML parser. Requires the XML, JavaScript, and CSS
+ parsers to also be loaded, so your <code>parserfile</code>
+ option looks something like <code>["parsexml.js", "parsecss.js",
+ "tokenizejavascript.js", "parsejavascript.js",
+ "parsehtmlmixed.js"]</code>.</dd>
+
+ <dt><code><a
+ href="js/parsesparql.js">parsesparql.js</a></code></dt><dd>Parses
+ the <a href="http://en.wikipedia.org/wiki/SPARQL">SPARQL</a>
+ query language. Example styles in <code><a
+ href="css/sparqlcolors.css">css/sparqlcolors.css</a></code></dd>
+
+ </dl>
+
+ <h2 id="programming">Programming Interface</h2>
+
+ <p>To be as flexible as possible, CodeMirror implements a very
+ plain editable field, without any accompanying buttons, bells, and
+ whistles. <code>CodeMirror</code> objects do, however, provide a
+ number of methods that make it possible to add extra functionality
+ around the editor. <a
+ href="js/mirrorframe.js"><code>mirrorframe.js</code></a> provides a
+ basic example of their usage.</p>
+
+ <dl>
+
+ <dt><code>getCode()</code> →
+ <code>string</code></dt><dd>Returns the current content of the
+ editor, as a string.</dd>
+
+ <dt><code>setCode(string)</code></dt><dd>Replaces the current
+ content of the editor with the given value.</dd>
+
+ <dt><code>focus()</code></dt><dd>Gives focus to the editor
+ frame.</dd>
+
+ <dt><code>currentLine()</code> →
+ <code>number</code></dt><dd>Returns the line on which the cursor
+ is currently sitting. <span class="warn">(Deprecated, see the
+ line-based interface below)</span></dd>
+
+ <dt><code>jumpToLine(number)</code></dt><dd>Moves the cursor to
+ the start of the given line. <span
+ class="warn">(Deprecated)</span></dd>
+
+ <dt><code>selection()</code> →
+ <code>string</code></dt><dd>Returns the text that is currently
+ selected in the editor.</dd>
+
+ <dt><code>replaceSelection(string)</code></dt><dd>Replaces the
+ currently selected text with the given string. Will also cause
+ the editor frame to gain focus.</dd>
+
+ <dt><code>reindent()</code></dt><dd>Automatically re-indent the
+ whole document.</dd>
+
+ <dt><code>getSearchCursor(string, atCursor)</code> →
+ <code>cursor</code></dt><dd>The first argument indicates the
+ string that should be searched for, and the second indicates
+ whether searching should start at the cursor (<code>true</code>)
+ or at the start of the document (<code>false</code>). Returns an
+ object that provides an interface for searching. Call its
+ <code>findNext()</code> method to search for an occurrence of
+ the given string. This returns <code>true</code> if something is
+ found, or <code>false</code> if the end of document is reached.
+ When an occurrence has been found, you can call
+ <code>select()</code> to select it, or
+ <code>replace(string)</code> to replace it with a given string.
+ Note that letting the user change the document, or
+ programmatically changing it in any way except for calling
+ <code>replace</code> on the cursor itself, might cause a cursor
+ object to skip back to the beginning of the document.</dd>
+
+ <dt><code>undo()</code></dt><dd>Undo one changeset, if available.</dd>
+ <dt><code>redo()</code></dt><dd>Redo one changeset, if available.</dd>
+ <dt><code>historySize() → object</code></dt><dd>Get a
+ <code>{undo, redo}</code> object holding the sizes of the undo
+ and redo histories.</dd>
+
+ <dt><code>grabKeys(callback, filter)</code></dt><dd>Route
+ keyboard input in the editor to a callback function. This
+ function is given a slightly normalised (see
+ <code>normalizeEvent</code> in <a
+ href="js/util.js"><code>util.js</code></a>) <code>keydown</code>
+ event object. If a second argument is given, this will be used
+ to determine which events to apply the callback to. It should
+ take a key code (as in <code>event.keyCode</code>), and return a
+ boolean, where <code>true</code> means the event should be
+ routed to the callback, and <code>false</code> leaves the key to
+ perform its normal behaviour.</dd>
+ <dt><code>ungrabKeys()</code></dt><dd>Revert the effect of
+ <code>grabKeys</code>.</dd>
+ </dl>
+
+ <p>For detailed interaction with the content of the editor,
+ CodeMirror exposes a line-oriented interface, which allows you to
+ inspect and manipulate the document line by line. Line handles
+ should be considered opaque (they are in fact the <code>BR</code>
+ nodes at the start of the line), except that the value
+ <code>false</code> (but <em>not</em> <code>null</code>) always
+ denotes an invalid value. Since changing the document might cause
+ some line handles to become invalid, every function that takes
+ them as argument can throw
+ <code>CodeMirror.InvalidLineHandle</code>. These are the relevant
+ methods:</p>
+
+ <dl>
+ <dt><code>cursorPosition(start)</code> →
+ <code>object</code></dt><dd>Retrieve a <code>{line,
+ character}</code> object representing the cursor position.
+ <code>start</code> defaults to <code>true</code> and determines
+ if the startpoint or the endpoint of the selection is used.</dd>
+ <dt><code>firstLine()</code> →
+ <code>handle</code></dt><dd>Get the first line of the
+ document.</dd>
+ <dt><code>lastLine()</code> →
+ <code>handle</code></dt><dd>The last line.</dd>
+ <dt><code>nextLine(handle)</code> →
+ <code>handle</code></dt><dd>Get the line after the given one, or
+ <code>false</code> if that was the last line.</dd>
+ <dt><code>prevLine(handle)</code> →
+ <code>handle</code></dt><dd>Find the line before the given one,
+ return <code>false</code> if that was the first line.</dd>
+ <dt><code>nthLine(number)</code> →
+ <code>handle</code></dt><dd>Find the Nth line of the document.
+ Note that the first line counts as one, not zero. Returns
+ <code>false</code> if there is no such line.</dd>
+ <dt><code>lineContent(handle)</code> →
+ <code>string</code></dt><dd>Retrieve the content of the
+ line.</dd>
+ <dt><code>setLineContent(handle, string)</code></dt><dd>Replace
+ the content of the line with the given string.</dd>
+ <dt><code>lineNumber(handle)</code> →
+ <code>number</code></dt><dd>Ask which line of the document
+ (1-based) the given line is.</dd>
+ <dt><code>selectLines(startHandle, startOffset,
+ endHandle, endOffset)</code></dt><dd>Move the selection to a
+ specific point. <code>endHandle</code> and
+ <code>endOffset</code> can be omitted to just place the cursor
+ somewhere without selecting any text.</dd>
+ <dt><code>insertIntoLine(handle, position,
+ text)</code></dt><dd>Insert a piece of text into a line.
+ <code>position</code> can be an integer, specifying the position
+ in the line where the text should be inserted, or the string
+ <code>"end"</code>, for the end of the line.</dd>
+ </dl>
+
+ <h2 id="writeparser">Writing a Parser</h2>
+
+ <p>A parser is implemented by one or more files (see
+ <code>parserfile</code> above) which, when loaded, add a
+ <code>Parser</code> object to the <code>Editor</code> object
+ defined by <a href="js/editor.js"><code>editor.js</code></a>. This
+ object must support the following interface:</p>
+
+ <dl>
+
+ <dt><code>make(stream)</code></dt><dd>A function that, given a
+ string stream (see <a
+ href="js/stringstream.js"><code>stringstream.js</code></a>),
+ creates a parser. The behaviour of this parser is described
+ below.</dd>
+
+ <dt><code>electricChars</code></dt><dd>An optional string
+ containing the characters that, when typed, should cause the
+ indentation of the current line to be recomputed (for example
+ <code>"{}"</code> for c-like languages).</dd>
+
+ <dt><code>configure(object)</code></dt><dd>An optional function
+ that can be used to configure the parser. If it exists, and an
+ editor is given a <code>parserConfig</code> option, it will be
+ called with the value of that option.</dd>
+
+ <dt><code>firstIndentation(chars, current,
+ direction)</code></dt><dd>An optional function that is used to
+ determine the proper indentation of the first line of a
+ document. When not provided, <code>0</code> is used.</dd>
+ </dl>
+
+ <p>When the <code>make</code> method is called with a string
+ stream, it should return a MochiKit-style iterator: an object with
+ a <code>next</code> method, which will raise
+ <code>StopIteration</code> when it is at its end (see <a
+ href="http://bob.pythonmac.org/archives/2005/07/06/iteration-in-javascript/">this</a>
+ for details). This iterator, when called, will consume input from
+ the string stream, and produce a token object.</p>
+
+ <p>Token objects represent a single significant piece of the text
+ that is being edited. A token object must have a
+ <code>value</code> property holding the text it stands for, and a
+ <code>style</code> property with the CSS class that should be used
+ to colour this element. This can be anything, except that any
+ whitespace at the start of a line should <em>always</em> have
+ class <code>"whitespace"</code>: The editor must be able to
+ recognize these when it indents lines. Furthermore, each newline
+ character <em>must</em> have its own separate token, which has an
+ <code>indentation</code> property holding a function that can be
+ used to determine the proper indentation level for the next line.
+ This function optionally takes the text in the first token of the
+ next line, the current indentation of the line, and the
+ 'direction' of the indentation as arguments, which it can use to
+ adjust the indentation level. The direction argument is only
+ useful for modes in which lines do not have a fixed indentation,
+ and can be modified by multiple tab presses. It is
+ <code>null</code> for 'default' indentations (like what happens
+ when the user presses enter), <code>true</code> for regular tab
+ presses, and <code>false</code> for control-tab or shift-tab.</p>
+
+ <p>So far this should be pretty easy. The hard part is that this
+ iterator must also have a <code>copy</code> method. This method,
+ called without arguments, returns a function representing the
+ current state of the parser. When this state function is later
+ called with a string stream as its argument, it returns a parser
+ object that resumes parsing using the old state and the new input
+ stream. It may assume that only one parser is active at a time,
+ and can clobber the state of the old parser if it wants.</p>
+
+ <p>For examples, see <a
+ href="js/parsejavascript.js"><code>parsejavascript.js</code></a>,
+ <a href="js/parsexml.js"><code>parsexml.js</code></a>, and <a
+ href="js/parsecss.js"><code>parsecss.js</code></a>.</p>
+
+ </body>
+</html>
Added: lenya/branches/BRANCH_2_0_X/src/modules/editors/resources/codemirror/0.61/mixedtest.html
URL: http://svn.apache.org/viewvc/lenya/branches/BRANCH_2_0_X/src/modules/editors/resources/codemirror/0.61/mixedtest.html?rev=794002&view=auto
==============================================================================
--- lenya/branches/BRANCH_2_0_X/src/modules/editors/resources/codemirror/0.61/mixedtest.html (added)
+++ lenya/branches/BRANCH_2_0_X/src/modules/editors/resources/codemirror/0.61/mixedtest.html Tue Jul 14 18:14:46 2009
@@ -0,0 +1,52 @@
+<html xmlns="http://www.w3.org/1999/xhtml">
+ <head>
+ <script src="js/codemirror.js" type="text/javascript"></script>
+ <title>CodeMirror: HTML mixed-mode demonstration</title>
+ <link rel="stylesheet" type="text/css" href="css/docs.css"/>
+ </head>
+ <body style="padding: 20px;">
+
+<p>This is a simple demonstration of the HTML mixed-mode indentation
+module for <a href="index.html">CodeMirror</a>. Script tags use the JS
+parser, style tags use the CSS parser.</p>
+
+<div style="border: 1px solid black; padding: 3px;">
+<textarea id="code" cols="120" rows="30">
+<html>
+ <head>
+ <title>HTML Example</title>
+ <script type="text/javascript">
+ function foo(bar, baz) {
+ alert("quux");
+ return bar + baz + 1;
+ }
+ </script>
+ <style type="text/css">
+ div.border {
+ border: 1px solid black;
+ padding: 3px;
+ }
+ #foo code {
+ font-family: courier, monospace;
+ font-size: 80%;
+ color: #448888;
+ }
+ </style>
+ </head>
+ <body>
+ <p>Duh</p>
+ </body>
+</html>
+</textarea>
+</div>
+
+<script type="text/javascript">
+ var editor = CodeMirror.fromTextArea('code', {
+ height: "350px",
+ parserfile: ["parsexml.js", "parsecss.js", "tokenizejavascript.js", "parsejavascript.js", "parsehtmlmixed.js"],
+ stylesheet: ["css/xmlcolors.css", "css/jscolors.css", "css/csscolors.css"],
+ path: "js/"
+ });
+</script>
+ </body>
+</html>
Added: lenya/branches/BRANCH_2_0_X/src/modules/editors/resources/codemirror/0.61/sparqltest.html
URL: http://svn.apache.org/viewvc/lenya/branches/BRANCH_2_0_X/src/modules/editors/resources/codemirror/0.61/sparqltest.html?rev=794002&view=auto
==============================================================================
--- lenya/branches/BRANCH_2_0_X/src/modules/editors/resources/codemirror/0.61/sparqltest.html (added)
+++ lenya/branches/BRANCH_2_0_X/src/modules/editors/resources/codemirror/0.61/sparqltest.html Tue Jul 14 18:14:46 2009
@@ -0,0 +1,41 @@
+<html xmlns="http://www.w3.org/1999/xhtml">
+ <head>
+ <script src="js/codemirror.js" type="text/javascript"></script>
+ <title>CodeMirror: Sparql demonstration</title>
+ <link rel="stylesheet" type="text/css" href="css/docs.css"/>
+ </head>
+ <body style="padding: 20px;">
+
+<p>Demonstration of <a href="index.html">CodeMirror</a>'s Sparql
+highlighter.</p>
+
+<div class="border">
+<textarea id="code" cols="120" rows="30">
+PREFIX a: <http://www.w3.org/2000/10/annotation-ns#>
+PREFIX dc: <http://purl.org/dc/elements/1.1/>
+PREFIX foaf: <http://xmlns.com/foaf/0.1/>
+
+# Comment!
+
+SELECT ?given ?family
+WHERE {
+ ?annot a:annotates <http://www.w3.org/TR/rdf-sparql-query/> .
+ ?annot dc:creator ?c .
+ OPTIONAL {?c foaf:given ?given ;
+ foaf:family ?family } .
+ FILTER isBlank(?c)
+}
+</textarea>
+</div>
+
+<script type="text/javascript">
+ var editor = CodeMirror.fromTextArea('code', {
+ height: "250px",
+ parserfile: "parsesparql.js",
+ stylesheet: "css/sparqlcolors.css",
+ path: "js/"
+ });
+</script>
+
+ </body>
+</html>
Added: lenya/branches/BRANCH_2_0_X/src/modules/editors/resources/codemirror/0.61/story.html
URL: http://svn.apache.org/viewvc/lenya/branches/BRANCH_2_0_X/src/modules/editors/resources/codemirror/0.61/story.html?rev=794002&view=auto
==============================================================================
--- lenya/branches/BRANCH_2_0_X/src/modules/editors/resources/codemirror/0.61/story.html (added)
+++ lenya/branches/BRANCH_2_0_X/src/modules/editors/resources/codemirror/0.61/story.html Tue Jul 14 18:14:46 2009
@@ -0,0 +1,651 @@
+<html xmlns="http://www.w3.org/1999/xhtml">
+ <head>
+ <title>Implementing a syntax-higlighting JavaScript editor in JavaScript</title>
+ <style type="text/css">
+ body {
+ padding: 3em 6em;
+ max-width: 50em;
+ }
+ h1 {
+ text-align: center;
+ margin: 0;
+ }
+ h2 {
+ font-size: 130%;
+ }
+ code {
+ font-family: courier, monospace;
+ font-size: 80%;
+ color: #144;
+ }
+ p {
+ margin: 1em 0;
+ }
+ pre.code {
+ margin: 1.1em 12px;
+ border: 1px solid #CCCCCC;
+ padding: .4em;
+ font-family: courier, monospace;
+ }
+ </style>
+ <link rel="stylesheet" type="text/css" href="css/jscolors.css"/>
+ </head>
+ <body>
+ <h1 style="font-size: 180%;">Implementing a syntax-higlighting JavaScript editor in JavaScript</h1>
+ <h1 style="font-size: 110%;">or</h1>
+ <h1 style="font-size: 130%; margin-bottom: 3em;">A brutal odyssey to the dark side of the DOM tree</h1>
+
+ <p style="font-size: 80%">
+ <b>Topic</b>: JavaScript, advanced browser weirdness, cool programming techniques<br/>
+ <b>Audience</b>: Programmers, especially JavaScript programmers<br/>
+ <b>Author</b>: Marijn Haverbeke<br/>
+ <b>Date</b>: May 24th 2007
+ </p>
+
+ <p style="color: #811; font-size: 90%; font-style: italic">Note: some of the details given here no
+ longer apply to the current <a
+ href="http://marijn.haverbeke.nl/codemirror">CodeMirror</a>
+ codebase, which has evolved quite a bit in the meantime.</p>
+
+ <p>In one of his (very informative) <a
+ href="http://video.yahoo.com/?t=t&p=douglascrockford">video
+ lectures</a>, Douglas Crockford remarks that writing JavaScript for
+ the web is 'programming in a hostile environment'. I had done my fair
+ share of weird workarounds, and even occasonally gave up an on idea
+ entirely because browsers just wouldn't support it, but before this
+ project I never really realized just how powerless a programmer can be
+ in the face of buggy, incompatible, and poorly designed platforms.</p>
+
+ <p>The plan was not ridiculously ambitious. I wanted to 'enhance' a
+ textarea to the point where writing code in it is pleasant. This meant
+ automatic indentation and, if possible at all, syntax highlighting.</p>
+
+ <p>In this document I describe the story of implementing this, for your
+ education and amusement. A demonstration of the resulting program,
+ along with the source code, can be found at <a
+ href="http://marijn.haverbeke.nl/codemirror">my website</a>.</p>
+
+ <h2>Take one: Only indentation</h2>
+
+ <p>The very first attempt merely added auto-indentation to a textarea
+ element. It would scan backwards through the content of the area,
+ starting from the cursor, until it had enough information to decide
+ how to indent the current line. It took me a while to figure out a
+ decent model for indenting JavaScript code, but in the end this seems
+ to work:</p>
+
+ <ul>
+ <li>Code that sits inside a block is indented one unit (generally two
+ spaces) more than the statement or brace that opened the block.</li>
+ <li>A statement that is continued to the next line is indented one unit
+ more than the line that starts the statement.</li>
+ <li>When dealing with lists of arguments or the content of array and
+ object literals there are two possible models. If there is any text
+ directly after the opening brace, bracket, or parenthesis,
+ subsequent lines are aligned with this opening character. If the
+ opening character is followed by a newline (optionally with whitespace
+ or comments before it), the next line is indented one unit further
+ than the line that started the list.</li>
+ <li>And, obviously, if a statement follows another statement it is
+ indented the same amount as the one before it.</li>
+ </ul>
+
+ <p>When scanning backwards through code one has to take string values,
+ comments, and regular expressions (which are delimited by slashes)
+ into account, because braces and semicolons and such are not
+ significant when they appear inside them. Single-line ('//') comments
+ turned out to be rather inefficient to check for when doing a
+ backwards scan, since every time you encounter a newline you have to
+ go on to the next newline to determine whether this line ends in a
+ comment or not. Regular expressions are even worse ― without
+ contextual information they are impossible to distinguish from the
+ division operator, and I didn't get them working in this first
+ version.</p>
+
+ <p>To find out which line to indent, and to make sure that adding or
+ removing whitespace doesn't cause the cursor to jump in strange ways,
+ it is necessary to determine which text the user has selected. Even
+ though I was working with just a simple textarea at this point, this
+ was already a bit of a headache.</p>
+
+ <p>On W3C-standards-respecting browsers, textarea nodes have
+ <code>selectionStart</code> and <code>selectionEnd</code>
+ properties which nicely give you the amount of characters before
+ the start and end of the selection. Great!</p>
+
+ <p>Then, there is Internet Explorer. Internet Explorer also has an API
+ for looking at and manipulating selections. It gives you information
+ such as a detailed map of the space the selected lines take up on the
+ screen, in pixels, and of course the text inside the selection. It
+ does, however, not give you much of a clue on where the selection is
+ located in the document.</p>
+
+ <p>After some experimentation I managed to work out an elaborate
+ method for getting something similar to the
+ <code>selectionStart</code> and <code>selectionEnd</code> values
+ in other browsers. It worked like this:</p>
+
+ <ul>
+ <li>Get the <code>TextRange</code> object corresponding to the selection.</li>
+ <li>Record the length of the text inside it.</li>
+ <li>Make another <code>TextRange</code> that covers the whole textarea element.</li>
+ <li>Set the start of the first <code>TextRange</code> to the start of the second one.</li>
+ <li>Again get the length of the text in the first object.</li>
+ <li>Now <code>selectionEnd</code> is the second length, and <code>selectionStart</code> is
+ the second minus the first one.</li>
+ </ul>
+
+ <p>That seemed to work, but when resetting the selection after modifying
+ the content of the textarea I ran into another interesting feature of
+ these <code>TextRange</code>s: You can move their endpoints by a given number of
+ characters, which is useful when trying to set a cursor at the Nth
+ character of a textarea, but in this context, newlines are <em>not</em>
+ considered to be characters, so you'll always end up one character too
+ far for every newline you passed. Of course, you can count newlines
+ and compensate for this (though it is still not possible to position
+ the cursor right in front of a newline). Sheesh.</p>
+
+ <p>After ragging on Internet Explorer for a while, let us move on and rag
+ on Firefox a bit. It turns out that, in Firefox, getting and setting
+ the text content of a DOM element is unexplainably expensive,
+ especially when there is a lot of text involved. As soon as I tried to
+ use my indentation code to indent itself (some 400 lines), I found
+ myself waiting for over four seconds every time I pressed enter. That
+ seemed a little slow.</p>
+
+ <h2>designMode it is</h2>
+
+ <p>The solution was obvious: Since the text inside a textarea can only be
+ manipulated as one single big string, I had to spread it out over
+ multiple nodes. How do you spread editable content over multiple
+ nodes? Right! <code>designMode</code> or <code>contentEditable</code>.</p>
+
+ <p>Now I wasn't entirely naive about <code>designMode</code>, I had been looking
+ into writing a non-messy WYSIWYG editor before, and at that time I had
+ concluded two things:</p>
+
+ <ul>
+ <li>It is impossible to prevent the user from inserting whichever HTML
+ junk he wants into the document.</li>
+ <li>In Internet Explorer, it is extemely hard to get a good view
+ on what nodes the user has selected.</li>
+ </ul>
+
+ <p>Basically, the good folks at Microsoft designed a really bad interface
+ for putting editable documents in pages, and the other browsers, not
+ wanting to be left behind, more or less copied that. And there isn't
+ much hope for a better way to do this appearing anytime soon. Wise
+ people probably use a Flash movie or (God forbid) a Java applet for
+ these kind of things, though those are not without drawbacks either.</p>
+
+ <p>Anyway, seeing how using an editable document would also make syntax
+ highlighting possible, I foolishly went ahead. There is something
+ perversely fascinating about trying to build a complicated system on a
+ lousy, unsuitable platform.</p>
+
+ <h2>A parser</h2>
+
+ <p>How does one do decent syntax highlighting? A very simple scanning can
+ tell the difference between strings, comments, keywords, and other
+ code. But this time I wanted to actually be able to recognize regular
+ expressions, so that I didn't have any blatant incorrect behaviour
+ anymore.</p>
+
+ <p>That brought me to the idea of doing a serious parse on the code. This
+ would not only make detecting regular expressions much easier, it
+ would also give me detailed information about the code, which can be
+ used to determine proper indentation levels, and to make subtle
+ distinctions in colouring, for example the difference between variable
+ names and property names.</p>
+
+ <p>And hey, when we're parsing the whole thing, it would even be possible
+ to make a distinction between local and global variables, and colour
+ them differently. If you've ever programmed JavaScript you can
+ probably imagine how useful this would be ― it is ridiculously easy
+ to accidentally create global instead of local variables. I don't
+ consider myself a JavaScript rookie anymore, but it was (embarrasingly
+ enough) only this week that I realized that my habit of typing <code>for
+ (name in object) ...</code> was creating a global variable <code>name</code>, and that
+ I should be typing <code>for (var name in object) ...</code> instead.</p>
+
+ <p>Re-parsing all the code the user has typed in every time he hits a key
+ is obviously not feasible. So how does one combine on-the-fly
+ highlighting with a serious parser? One option would be to split the
+ code into top-level statements (functions, variable definitions, etc.)
+ and parse these separately. This is horribly clunky though, especially
+ considering the fact that modern JavaScripters often put all the code
+ in a file in a single big object or function to prevent namespace
+ pollution.</p>
+
+ <p>I have always liked continuation-passing style and generators. So the
+ idea I came up with is this: An interruptable, resumable parser. This
+ is a parser that does not run through a whole document at once, but
+ parses on-demand, a little bit at a time. At any moment you can create
+ a copy of its current state, which can be resumed later. You start
+ parsing at the top of the code, and keep going as long as you like,
+ but throughout the document, for example at every end of line, you
+ store a copy of the current parser state. Later on, when line 106
+ changes, you grab the interrupted parser that was stored at the end of
+ line 105, and use it to re-parse line 106. It still knows exactly what
+ the context was at that point, which local variables were defined,
+ which unfinished statements were encountered, and so on.</p>
+
+ <p>But that, unfortunately, turned out to be not quite as easy as it
+ sounds.</p>
+
+ <h2>The DOM nodes underfoot</h2>
+
+ <p>Of course, when working inside an editable frame we don't just
+ have to deal with text. The code will be represented by some kind
+ of DOM tree. My first idea was to set the <code>white-space:
+ pre</code> style for the frame and try to work with mostly text,
+ with the occasional coloured <code>span</code> element. It turned
+ out that support for <code>white-space: pre</code> in browsers,
+ especially in editable frames, is so hopelessly glitchy that this
+ was unworkable.</p>
+
+ <p>Next I tried a series of <code>div</code> elements, one per
+ line, with <code>span</code> elements inside them. This seemed to
+ nicely reflect the structure of the code in a shallowly
+ hierarchical way. I soon realized, however, that my code would be
+ much more straightfoward when using no hierarchy whatsoever
+ ― a series of <code>span</code>s, with <code>br</code> tags
+ at the end of every line. This way, the DOM nodes form a flat
+ sequence that corresponds to the sequence of the text ―
+ just extract text from <code>span</code> nodes and substitute
+ newlines for <code>br</code> nodes.</p>
+
+ <p>It would be a shame if the editor would fall apart as soon as
+ someone pastes some complicated HTML into it. I wanted it to be
+ able to deal with whatever mess it finds. This means using some
+ kind of HTML-normalizer that takes arbitrary HTML and flattens it
+ into a series of <code>br</code>s and <code>span</code> elements
+ that contain a single text node. Just like the parsing process, it
+ would be best if this did not have to done to the entire buffer
+ every time something changes.</p>
+
+ <p>It took some banging my head against my keyboard, but I found a very
+ nice way to model this. It makes heavy use of generators, for which I
+ used <a href="http://www.mochikit.com">MochiKit</a>'s iterator
+ framework. Bob Ippolito explains the concepts in this library very
+ well in his <a
+ href="http://bob.pythonmac.org/archives/2005/07/06/iteration-in-javascript/">blog
+ post</a> about it. (Also notice some of the dismissive comments at the
+ bottom of that post. They say "I don't think I really want to learn
+ this, so I'll make up some silly reason to condemn it.")</p>
+
+ <p>The highlighting process consists of the following elements:
+ normalizing the DOM tree, extracting the text from the DOM tree,
+ tokenizing this text, parsing the tokens, and finally adjusting the
+ DOM nodes to reflect the structure of the code.</p>
+
+ <p>The first two, I put into a single generator. It scans the DOM
+ tree, fixing anything that is not a simple top-level
+ <code>span</code> or <code>br</code>, and it produces the text
+ content of the nodes (or a newline in case of a <code>br</code>)
+ as its output ― each time it is called, it yields a string.
+ Continuation passing style was a good way to model this process in
+ an iterator, which has to be processed one step at a time. Look at
+ this simplified version:</p>
+
+ <pre class="code"><span class="js-keyword">function</span> <span class="js-variable">traverseDOM</span>(<span class="js-variabledef">start</span>){
+ <span class="js-keyword">var</span> <span class="js-variabledef">cc</span> = <span class="js-keyword">function</span>(){<span class="js-variable">scanNode</span>(<span class="js-localvariable">start</span>, <span class="js-variable">stop</span>);};
+ <span class="js-keyword">function</span> <span class="js-variabledef">stop</span>(){
+ <span class="js-localvariable">cc</span> = <span class="js-localvariable">stop</span>;
+ <span class="js-keyword">throw</span> <span class="js-variable">StopIteration</span>;
+ }
+ <span class="js-keyword">function</span> <span class="js-variabledef">yield</span>(<span class="js-variabledef">value</span>, <span class="js-variabledef">c</span>){
+ <span class="js-localvariable">cc</span> = <span class="js-localvariable">c</span>;
+ <span class="js-keyword">return</span> <span class="js-localvariable">value</span>;
+ }
+
+ <span class="js-keyword">function</span> <span class="js-variabledef">scanNode</span>(<span class="js-variabledef">node</span>, <span class="js-variabledef">c</span>){
+ <span class="js-keyword">if</span> (<span class="js-localvariable">node</span>.<span class="js-property">nextSibling</span>)
+ <span class="js-keyword">var</span> <span class="js-variabledef">nextc</span> = <span class="js-keyword">function</span>(){<span class="js-localvariable">scanNode</span>(<span class="js-localvariable">node</span>.<span class="js-property">nextSibling</span>, <span class="js-localvariable">c</span>);};
+ <span class="js-keyword">else</span>
+ <span class="js-keyword">var</span> <span class="js-variabledef">nextc</span> = <span class="js-localvariable">c</span>;
+
+ <span class="js-keyword">if</span> (<span class="js-comment">/* node is proper span element */</span>)
+ <span class="js-keyword">return</span> <span class="js-localvariable">yield</span>(<span class="js-localvariable">node</span>.<span class="js-property">firstChild</span>.<span class="js-property">nodeValue</span>, <span class="js-localvariable">nextc</span>);
+ <span class="js-keyword">else</span> <span class="js-keyword">if</span> (<span class="js-comment">/* node is proper br element */</span>)
+ <span class="js-keyword">return</span> <span class="js-localvariable">yield</span>(<span class="js-string">"\n"</span>, <span class="js-localvariable">nextc</span>);
+ <span class="js-keyword">else</span>
+ <span class="js-comment">/* flatten node, yield its textual content */</span>;
+ }
+
+ <span class="js-keyword">return</span> {<span class="js-property">next</span>: <span class="js-keyword">function</span>(){<span class="js-keyword">return</span> <span class="js-localvariable">cc</span>();}};
+}</pre>
+
+ <p>The variable <code>c</code> stands for 'continuation', and <code>cc</code> for 'current
+ continuation' ― that last variable is used to store the function to
+ continue with, when yielding a value to the outside world. Every time
+ control leaves this function, it has to make sure that <code>cc</code> is set to
+ a suitable value, which is what <code>yield</code> and <code>stop</code> take care of.</p>
+
+ <p>The object that is returned contains a <code>next</code> method, which is
+ MochiKit's idea of an iterator, and the initial continuation just
+ throws a <code>StopIteration</code>, which is how MochiKit signals that an
+ iterator has reached its end.</p>
+
+ <p>The first lines of <code>scanNode</code> extend the continuation with the task of
+ scanning the next node, if there is a next node. The rest of the
+ function decides what kind of value to <code>yield</code>. Note that this is a
+ rather trivial example of this technique, since the process of going
+ through these nodes is basically linear (it was much, much more
+ complex in earlier versions), but still the trick with the
+ continuations makes the code shorter and, for those in the know,
+ clearer than the equivalent 'storing the iterator state in variables'
+ approach.</p>
+
+ <p>The next iterator that the input passes through is the
+ tokenizer. Well, actually, there is another iterator in between
+ that isolates the tokenizer from the fact that the DOM traversal
+ yields a bunch of separate strings, and presents them as a single
+ character stream (with a convenient <code>peek</code> operation),
+ but this is not a very interesting one. What the tokenizer returns
+ is a stream of token objects, each of which has a
+ <code>value</code>, its textual content, a <code>type</code>, like
+ <code>"variable"</code>, <code>"operator"</code>, or just itself,
+ <code>"{"</code> for example, in the case of significant
+ punctuation or special keywords. They also have a
+ <code>style</code>, which is used later by the highlighter to give
+ their <code>span</code> elements a class name (the parser will
+ still adjust this in some cases).</p>
+
+ <p>At first I assumed the parser would have to talk back to the
+ tokenizer about the current context, in order to be able to
+ distinguish those accursed regular expressions from divisions, but
+ it seems that regular expressions are only allowed if the previous
+ (non-whitespace, non-comment) token was either an operator, a
+ keyword like <code>new</code> or <code>throw</code>, or a specific
+ kind of punctuation (<code>"[{}(,;:"</code>) that indicates a new
+ expression can be started here. This made things considerably
+ easier, since the 'regexp or no regexp' question could stay
+ entirely within the tokenizer.</p>
+
+ <p>The next step, then, is the parser. It does not do a very
+ thorough job because, firstly, it has to be fast, and secondly, it
+ should not go to pieces when fed an incorrect program. So only
+ superficial constructs are recognized, keywords that resemble each
+ other in syntax, such as <code>while</code> and <code>if</code>,
+ are treated in precisely the same way, as are <code>try</code> and
+ <code>else</code> ― the parser doesn't mind if an
+ <code>else</code> appears without an <code>if</code>. Stuff that
+ binds variables, <code>var</code>, <code>function</code>, and
+ <code>catch</code> to be precise, is treated with more care,
+ because the parser wants to know about local variables.</p>
+
+ <p>Inside the parser, three kinds of context are stored. Firstly, a set
+ of known local variables, which is used to adjust the style of
+ variable tokens. Every time the parser enters a function, a new set of
+ variables is created. If there was already such a set (entering an
+ inner function), a pointer to the old one is stored in the new one. At
+ the end of the function, the current variable set is 'popped' off and
+ the previous one is restored.</p>
+
+ <p>The second kind of context is the lexical context, this keeps track of
+ whether we are inside a statement, block, or list. Like the variable
+ context, it also forms a stack of contexts, with each one containing a
+ pointer to the previous ones so that they can be popped off again when
+ they are finished. This information is used for indentation. Every
+ time the parser encounters a newline token, it attaches the current
+ lexical context and a 'copy' of itself (more about that later) to this
+ token.</p>
+
+ <p>The third context is a continuation context. This parser does not use
+ straight continuation style, instead it uses a stack of actions that
+ have to be performed. These actions are simple functions, a kind of
+ minilanguage, they act on tokens, and decide what kind of new actions
+ should be pushed onto the stack. Here are some examples:</p>
+
+<pre class="code"><span class="js-keyword">function</span> <span class="js-variable">expression</span>(<span class="js-variabledef">type</span>){
+ <span class="js-keyword">if</span> (<span class="js-localvariable">type</span> in <span class="js-variable">atomicTypes</span>) <span class="js-variable">cont</span>(<span class="js-variable">maybeoperator</span>);
+ <span class="js-keyword">else</span> <span class="js-keyword">if</span> (<span class="js-localvariable">type</span> == <span class="js-string">"function"</span>) <span class="js-variable">cont</span>(<span class="js-variable">functiondef</span>);
+ <span class="js-keyword">else</span> <span class="js-keyword">if</span> (<span class="js-localvariable">type</span> == <span class="js-string">"("</span>) <span class="js-variable">cont</span>(<span class="js-variable">pushlex</span>(<span class="js-string">"list"</span>), <span class="js-variable">expression</span>, <span class="js-variable">expect</span>(<span class="js-string">")"</span>), <span class="js-variable">poplex</span>);
+ <span class="js-keyword">else</span> <span class="js-keyword">if</span> (<span class="js-localvariable">type</span> == <span class="js-string">"operator"</span>) <span class="js-variable">cont</span>(<span class="js-variable">expression</span>);
+ <span class="js-keyword">else</span> <span class="js-keyword">if</span> (<span class="js-localvariable">type</span> == <span class="js-string">"["</span>) <span class="js-variable">cont</span>(<span class="js-variable">pushlex</span>(<span class="js-string">"list"</span>), <span class="js-variable">commasep</span>(<span class="js-variable">expression</span>), <span class="js-variable">expect</span>(<span class="js-string">"]"</span>), <span class="js-variable">poplex</span>);
+ <span class="js-keyword">else</span> <span class="js-keyword">if</span> (<span class="js-localvariable">type</span> == <span class="js-string">"{"</span>) <span class="js-variable">cont</span>(<span class="js-variable">pushlex</span>(<span class="js-string">"list"</span>), <span class="js-variable">commasep</span>(<span class="js-variable">objprop</span>), <span class="js-variable">expect</span>(<span class="js-string">"}"</span>), <span class="js-variable">poplex</span>);
+ <span class="js-keyword">else</span> <span class="js-keyword">if</span> (<span class="js-localvariable">type</span> == <span class="js-string">"keyword c"</span>) <span class="js-variable">cont</span>(<span class="js-variable">expression</span>);
+}
+
+<span class="js-keyword">function</span> <span class="js-variable">block</span>(<span class="js-variabledef">type</span>){
+ <span class="js-keyword">if</span> (<span class="js-localvariable">type</span> == <span class="js-string">"}"</span>) <span class="js-variable">cont</span>();
+ <span class="js-keyword">else</span> <span class="js-variable">pass</span>(<span class="js-variable">statement</span>, <span class="js-variable">block</span>);
+}</pre>
+
+ <p>The function <code>cont</code> (for continue), will push the actions it is given
+ onto the stack (in reverse order, so that the first one will be popped
+ first). Actions such as <code>pushlex</code> and <code>poplex</code> merely adjust the
+ lexical environment, while others, such as <code>expression</code> itself, do
+ actual parsing. <code>pass</code>, as seen in <code>block</code>, is similar to <code>cont</code>, but
+ it does not 'consume' the current token, so the next action will again
+ see this same token. In <code>block</code>, this happens when the function
+ determines that we are not at the end of the block yet, so it pushes
+ the <code>statement</code> function which will interpret the current token as the
+ start of a statement.</p>
+
+ <p>These actions are called by a 'driver' function, which filters out the
+ whitespace and comments, so that the parser actions do not have to
+ think about those, and keeps track of some things like the indentation
+ of the current line and the column at which the current token ends,
+ which are stored in the lexical context and used for indentation.
+ After calling an action, if the action called <code>cont</code>, this driver
+ function will return the current token, if <code>pass</code> (or nothing) was
+ called, it will immediately continue with the next action.</p>
+
+ <p>This goes to show that it is viable to write a quite elaborate
+ minilanguage in a macro-less language like JavaScript. I don't think
+ it would be possible to do something like this without closures (or
+ similarly powerful abstraction) though, I've certainly never seen
+ anything like it in Java code.</p>
+
+ <p>The way a 'copy' of the parser was produced shows a nice usage
+ of closures. Like with the DOM transformer shown above, most of
+ the local state of the parser is held in a closure produced by
+ calling <code>parse(stream)</code>. The function
+ <code>copy</code>, which is local to the parser function, produces
+ a new closure, with copies of all the relevant variables:</p>
+
+<pre class="code"><span class="js-keyword">function</span> <span class="js-variable">copy</span>(){
+ <span class="js-keyword">var</span> <span class="js-variabledef">_context</span> = <span class="js-variable">context</span>, <span class="js-variabledef">_lexical</span> = <span class="js-variable">lexical</span>, <span class="js-variabledef">_actions</span> = <span class="js-variable">copyArray</span>(<span class="js-variable">actions</span>);
+
+ <span class="js-keyword">return</span> <span class="js-keyword">function</span>(<span class="js-variabledef">_tokens</span>){
+ <span class="js-variable">context</span> = <span class="js-localvariable">_context</span>;
+ <span class="js-variable">lexical</span> = <span class="js-localvariable">_lexical</span>;
+ <span class="js-variable">actions</span> = <span class="js-variable">copyArray</span>(<span class="js-localvariable">_actions</span>);
+ <span class="js-variable">tokens</span> = <span class="js-localvariable">_tokens</span>;
+ <span class="js-keyword">return</span> <span class="js-variable">parser</span>;
+ };
+}</pre>
+
+ <p>Where <code>parser</code> is the object that contains the <code>next</code> (driver)
+ function, and a reference to this <code>copy</code> function. When the function
+ that <code>copy</code> produces is called with a token stream as argument, it
+ updates the local variables in the parser closure, and returns the
+ corresponding iterator object.</p>
+
+ <p>Moving on, we get to the last stop in this chain of generators, the
+ actual highlighter. You can view this one as taking two streams as
+ input, on the one hand there is the stream of tokens from the parser,
+ and on the other hand there is the DOM tree as left by the DOM
+ transformer. If everything went correctly, these two should be
+ synchronized. The highlighter can look at the current token, see if
+ the <code>span</code> in the DOM tree corresponds to it (has the same text
+ content, and the correct class), and if not it can chop up the DOM
+ nodes to conform to the tokens.</p>
+
+ <p>Every time the parser yields a newline token, the highligher
+ encounters a <code>br</code> element in the DOM stream. It takes the copy of the
+ parser and the lexical context from this token and attaches them to
+ the DOM node. This way, a new highlighting process can be started from
+ that node by re-starting the copy of the parser with a new token
+ stream, which reads tokens from the DOM nodes starting at that <code>br</code>
+ element, and the indentation code can use the lexical context
+ information to determine the correct indentation at that point.</p>
+
+ <h2>Selection woes</h2>
+
+ <p>All the above can be done using the DOM interface that all major
+ browsers have in common, and which is relatively free of weird bugs
+ and abberrations. However, when the user is typing in new code, this
+ must also be highlighted. For this to happen, the program must know
+ where the cursor currently is, and because it mucks up the DOM tree,
+ it has to restore this cursor position after doing the highlighting.</p>
+
+ <p>Re-highlighting always happens per line, because the copy of the
+ parser is stored only at the end of lines. Doing this every time the
+ user presses a key is terribly slow and obnoxious, so what I did was
+ keep a list of 'dirty' nodes, and as soon as the user didn't type
+ anyting for 300 milliseconds the program starts re-highlighting these
+ nodes. If it finds more than ten lines must be re-parsed, it does only
+ ten and waits another 300 milliseconds before it continues, this way
+ the browser never freezes up entirely.</p>
+
+ <p>As mentioned earlier, Internet Explorer's selection model is not the
+ most practical one. My attempts to build a wrapper that makes it look
+ like the W3C model all stranded. In the end I came to the conclusion
+ that I only needed two operations:</p>
+
+ <ul>
+ <li>Creating a selection 'snapshot' that can be restored after
+ highlighting, in such a way that it still works if some of the nodes
+ that were selected are replaced by other nodes with the same
+ size but a different structure.</li>
+ <li>Finding the top-level node around or before the cursor, to mark it
+ dirty or to insert indentation whitespace at the start of that line.</li>
+ </ul>
+
+ <p>It turns out that the pixel-based selection model that Internet
+ Explorer uses, which always seemed completely ludricrous to me, is
+ perfect for the first case. Since the DOM transformation (generally)
+ does not change the position of things, storing the pixel offsets of
+ the selection makes it possible to restore that same selection, never
+ mind what happened to the underlying DOM structure.</p>
+
+ <p>[Later addition: Note that this, due to the very random design
+ of the <a
+ href="http://msdn2.microsoft.com/en-us/library/ms535872(VS.85).aspx#">TextRange
+ interface</a>, only really works when the whole selection falls
+ within the visible part of the document.]</p>
+
+ <p>Doing the same with the W3C selection model is a lot harder. What I
+ ended up with was this:</p>
+
+ <ul>
+ <li>Create an object pointing to the nodes at the start and end of the
+ selection, and the offset within those nodes. This is basically the
+ information that the <code>Range</code> object gives you.</li>
+ <li>Make references from these nodes back to that object.</li>
+ <li>When replacing (part of) a node with another one, check for such a
+ reference, and when it is present, check whether this new node will
+ get the selection. If it does, move the reference from the old to the
+ new node, if it does not, adjust the offset in the selection object to
+ reflect the fact that part of the old node has been replaced.</li>
+ </ul>
+
+ <p>Now in the second case (getting the top-level node at the
+ cursor) the Internet Explorer cheat does not work. In the W3C
+ model this is rather easy, you have to do some creative parent-
+ and sibling-pointer following to arrive at the correct top-level
+ node, but nothing weird. In Internet Explorer, all we have to go
+ on is the <code>parentElement</code> method on a
+ <code>TextRange</code>, which gives the first element that
+ completely envelops the selection. If the cursor is inside a text
+ node, this is good, that text node tells us where we are. If the
+ cursor is between nodes, for example between two <code>br</code>
+ nodes, you get to top-level node itself back, which is remarkably
+ useless. In cases like this I stoop to a rather ugly hack (which
+ fortunately turned out to be acceptably fast) ― I create a
+ temporary empty <code>span</code> with an ID inside the selection,
+ get a reference to this <code>span</code> by ID, take its
+ <code>previousSibling</code>, and remove it again.</p>
+
+ <p>Unfortunately, Opera's selection implementation is buggy, and it
+ will give wildly incorrect <code>Range</code> objects when the cursor
+ is between two nodes. This is a bit of a showstopper, and until I find
+ a workaround for that or it gets fixed, the highlighter doesn't work
+ properly in Opera.</p>
+
+ <p>Also, when one presses enter in a <code>designMode</code>
+ document in Firefox or Opera, a <code>br</code> tag is inserted.
+ In Internet Explorer, pressing enter causes some maniacal gnome to
+ come out and start wrapping all the content before and after the
+ cursor in <code>p</code> tags. I suppose there is something to be
+ said for that, in principle, though if you saw the tag soup of
+ <code>font</code>s and nested paragraphs Internet Explorer
+ generates you would soon enough forget all about principle.
+ Anyway, getting unwanted <code>p</code> tags slowed the
+ highlighter down terribly ― it had to overhaul the whole
+ DOM tree to remove them again, every time the user pressed enter.
+ Fortunately I could fix this by capturing the enter presses and
+ manually inserting a <code>br</code> tag at the cursor.</p>
+
+ <p>On the subject of Internet Explorer's tag soup, here is an interesting
+ anecdote: One time, when testing the effect that modifying the content
+ of a selection had, I inspected the DOM tree and found a <code>"/B"</code>
+ element. This was not a closing tag, there are no closing tags in the
+ DOM tree, just elements. The <code>nodeName</code> of this element was actually
+ <code>"/B"</code>. That was when I gave up any notions of ever understanding the
+ profound mystery that is Internet Explorer.</p>
+
+ <h2>Closing thoughts</h2>
+
+ <p>Well, I despaired at times, but I did end up with a working JavaScript
+ editor. I did not keep track of the amount of time I wasted on this,
+ but I would estimate it to be around fifty hours. Finding workarounds
+ for browser bugs can be a terribly nonlinear process. I just spent
+ half a day working on a weird glitch in Firefox that caused the cursor
+ in the editable frame to be displayed 3/4 line too high when it was at
+ the very end of the document. Then I found out that setting the
+ style.display of the iframe to "block" fixed this (why not?). I'm
+ amazed how often issues that seem hopeless do turn out to be
+ avoidable, even if it takes hours of screwing around and some truly
+ non-obvious ideas.</p>
+
+ <p>For a lot of things, JavaScript + DOM elements are a surprisingly
+ powerful platform. Simple interactive documents and forms can be
+ written in browsers with very little effort, generally less than with
+ most 'traditional' platforms (Java, Win32, things like WxWidgets).
+ Libraries like Dojo (and a similar monster I once wrote myself) even
+ make complex, composite widgets workable. However, when applications
+ go sufficiently beyond the things that browsers were designed for, the
+ available APIs do not give enough control, are nonstandard and buggy,
+ and are often poorly designed. Because of this, writing such
+ applications, when it is even possible, is <em>painful</em> process.</p>
+
+ <p>And who likes pain? Sure, when finding that crazy workaround,
+ subdueing the damn browser, and getting everything to work, there
+ is a certain macho thrill. But one can't help wondering how much
+ easier things like preventing the user from pasting pictures in
+ his source code would be on another platform. Maybe something like
+ Silverlight or whatever other new browser plugin gizmos people are
+ pushing these days will become the way to solve things like this
+ in the future. But, personally, I would prefer for those browser
+ companies to put some real effort into things like cleaning up and
+ standardising shady things like <code>designMode</code>, fixing
+ their bugs, and getting serious about ECMAScript 4.</p>
+
+ <p>Which is probably not realistically going to happen anytime soon.</p>
+
+ <hr/>
+
+ <p>Some interesting projects similar to this:</p>
+
+ <ul>
+ <li><a href="http://gpl.internetconnection.net/vi/">vi clone</a></li>
+ <li><a href="http://robrohan.com/projects/9ne/">Emacs clone</a></li>
+ <li><a href="http://codepress.sourceforge.net/">CodePress</a></li>
+ <li><a href="http://www.codeide.com">CodeIDE</a></li>
+ <li><a href="http://www.cdolivet.net/editarea">EditArea</a></li>
+ </ul>
+
+ <hr/>
+
+ <p>If you have any remarks, criticism, or hints related to the
+ above, drop me an e-mail at <a
+ href="mailto:[email protected]">[email protected]</a>. If you say
+ something generally interesting, I'll include your reaction here
+ at the bottom of this page.</p>
+
+ </body>
+</html>
Modified: lenya/branches/BRANCH_2_0_X/src/modules/editors/resources/css/codemirror.css
URL: http://svn.apache.org/viewvc/lenya/branches/BRANCH_2_0_X/src/modules/editors/resources/css/codemirror.css?rev=794002&r1=794001&r2=794002&view=diff
==============================================================================
--- lenya/branches/BRANCH_2_0_X/src/modules/editors/resources/css/codemirror.css (original)
+++ lenya/branches/BRANCH_2_0_X/src/modules/editors/resources/css/codemirror.css Tue Jul 14 18:14:46 2009
@@ -1,4 +1,4 @@
-@import url(/modules/editors/codemirror/0.60/css/xmlcolors.css);
+@import url(/modules/editors/codemirror/0.61/css/xmlcolors.css);
.editbox {
font-size: 9pt;
Modified: lenya/branches/BRANCH_2_0_X/src/modules/editors/resources/javascript/sourceEditor/codemirror.js
URL: http://svn.apache.org/viewvc/lenya/branches/BRANCH_2_0_X/src/modules/editors/resources/javascript/sourceEditor/codemirror.js?rev=794002&r1=794001&r2=794002&view=diff
==============================================================================
--- lenya/branches/BRANCH_2_0_X/src/modules/editors/resources/javascript/sourceEditor/codemirror.js (original)
+++ lenya/branches/BRANCH_2_0_X/src/modules/editors/resources/javascript/sourceEditor/codemirror.js Tue Jul 14 18:14:46 2009
@@ -19,7 +19,7 @@
function insertEditor() {
editor = CodeMirror.fromTextArea('editorContent', {
- path: WEBAPP_BASE_URI + "modules/editors/codemirror/0.60/js/",
+ path: WEBAPP_BASE_URI + "modules/editors/codemirror/0.61/js/",
parserfile: "parsexml.js",
stylesheet: WEBAPP_BASE_URI + "modules/editors/css/codemirror.css",
height: "400px",
Modified: lenya/branches/BRANCH_2_0_X/src/modules/editors/usecases/forms/oneform.jx
URL: http://svn.apache.org/viewvc/lenya/branches/BRANCH_2_0_X/src/modules/editors/usecases/forms/oneform.jx?rev=794002&r1=794001&r2=794002&view=diff
==============================================================================
--- lenya/branches/BRANCH_2_0_X/src/modules/editors/usecases/forms/oneform.jx (original)
+++ lenya/branches/BRANCH_2_0_X/src/modules/editors/usecases/forms/oneform.jx Tue Jul 14 18:14:46 2009
@@ -33,7 +33,7 @@
<script type="text/javascript" src="/modules/editors/javascript/org.apache.lenya.editors.js"/>
<script type="text/javascript" src="/modules/editors/javascript/sourceEditor/lenya_glue.js"/>
<script type="text/javascript" src="/modules/editors/javascript/disablebackspace.js"/>
- <script type="text/javascript" src="/modules/editors/codemirror/0.60/js/codemirror.js"/>
+ <script type="text/javascript" src="/modules/editors/codemirror/0.61/js/codemirror.js"/>
<script type="text/javascript" src="/modules/editors/javascript/sourceEditor/codemirror.js"/>
<link rel="stylesheet" type="text/css" href="/modules/editors/css/sourceEditor.css"/>
</page:head>
lmpx.com only provides a reader for public news (NNTP) servers. It is not
affiliated with the servers or forums shown here and is not responsible for
the content of articles, which is written by their respective authors.