Proposal: ptl_compile.register_preprocessor()

Neil & friends,

I've been playing with my "extended PTL syntax" for a short while, and
have found it quite useful. Following discussion on the list, I
realize that it's best shared as a third-party package, rather than as
an addition to Quixote itself. But, currently, my module requires a
patch to ptl_compile. This is an obstacle to packaging my code as an
external product.

Would you be willing to add a hook to ptl_compile that would let me
register a preprocessor function? Something like:

-------------

--- ptl_compile.py~    Tue Jun 07 10:49:54 2005
+++ ptl_compile.py    Wed Aug 31 10:07:30 2005
@@ -185,7 +185,11 @@
                                              match.group('name'))
     return  _template_re.sub(replacement, buf)
 
+_external_preprocessors = []
+
 def parse(buf, filename='<string>'):
+    for ext_proc in _external_preprocessors:
+        buf = ext_proc(buf)
     buf = translate_tokens(buf)
     try:
         return TemplateTransformer().parsesuite(buf)
@@ -193,6 +197,14 @@
         # set the filename attribute
         raise SyntaxError(str(e), (filename, e.lineno, e.offset, e.text))
 
+def register_preprocessor(proc):
+    """
+    Registers a function that will be applied to the text of a PTL
+    file before it is processed by the PTL compiler. This provides a
+    a hook into the compilation process for external modules.
+    """
+    _external_preprocessors.append(proc)
+ 
 
 PTL_EXT = ".ptl"
 
-------------

Secondly, ptl_import would need a short patch, to ensure that the
ptl_compile is only loaded once:

--- ptl_import.py~    Wed Mar 16 14:59:28 2005
+++ ptl_import.py    Wed Aug 31 10:08:55 2005
@@ -17,7 +17,7 @@
 import marshal
 import __builtin__
 
-from ptl_compile import compile_template, PTL_EXT
+from quixote.ptl.ptl_compile import compile_template, PTL_EXT
 
 assert sys.hexversion >= 0x20000b1, "need Python 2.0b1 or later"
 
-------------

Then, my module could simply call:

    from quixote.ptl import ptl_compile

    def mytranslator(buf):
        ...
        return translated

    ptl_compile.register_preprocessor(mytranslator)


Since this code is only called at PTL compile-time, the overhead of
checking for preprocessors would be negligible. At worst, it clutters
the code a bit, but not much...

Thanks,

Graham