Re: Migration docutils 0.12 to 0.13

On Mon, Jan 2, 2017 at 3:26 PM, Jean Baptiste Favre
<[email protected]> wrote:
> Hello David,
>
> Thanks for your answer.
> Unfortunatly, the patch doesn't help.
>
> I still have a "KeyError: 'non_unescaped_whitespace_escape_before'" in
> docutils/parsers/rst/states.py at line 533.

In future, please send a full traceback. We need to know the context
(what called what).

> I guess it's the same kind of problem and I should redefine all local
> attributes (from lines 655 to 687) as class attributes as you did in the
> patch.

I think I see what the problem is now. docutils/parsers/rst/states.py,
Inliner.init_customizations contains the following line::

    args.update(vars(self.__class__))

The "vars" function gets the __dict__ from the object's class. But
you're using a subclass, so vars returns the __dict__ of the subclass,
without the the superclass's attributes, which is what is needed. It's
a bit of metaprogramming gone wrong. It's not safe for subclassing,
which is a bug. It's already a bit of a kludgey shortcut. We could
replace it with::

    args.update(vars(Inliner))

That's a bit smelly, but I see it's done elsewhere in the code. Try
making that change (patch attached).

We may have to explicitly refer to all the class attributes. I'll think on it.

David Goodger

> I'll test it asap and report,
> Cheers,
> Jean Baptiste
>
>
> On 02/01/2017 21:24, David Goodger wrote:
>> On Sun, Jan 1, 2017 at 4:17 PM, Jean Baptiste Favre
>> <[email protected]> wrote:
>>> Hello,
>>> I tried another workaround.
>>> Instead of using a custom class, I overrided init_cutomizations method
>>> using:
>>>
>>>   class Inliner(BaseInliner):
>>>     def __init(self):
>>>       BaseInliner.__init__(self)
>>>
>>>     def init_customizations(self, settings):
>>>       BaseInliner.init_customizations(self, settings)
>>>
>>>       issue_pattern = re.compile(u'''
>>>         {start_string_prefix}
>>>         TS-\d+
>>>         {end_string_suffix}'''.format(
>>>           start_string_prefix=self.start_string_prefix,
>>>           end_string_suffix=self.end_string_suffix),
>>>         re.VERBOSE | re.UNICODE)
>>>
>>>       self.implicit_dispatch.append((issue_pattern, self.issue_reference))
>>>
>>> I don't call explicitly init_customizations, but it's called: during
>>> run, I still get the error:
>>>
>>>   Exception occurred:
>>>     File
>>> "/usr/lib/python2.7/dist-packages/docutils/parsers/rst/states.py", line
>>> 530, in init_customizations
>>>         """ % args, re.VERBOSE | re.UNICODE),
>>>   KeyError: 'non_unescaped_whitespace_escape_before'
>>>   The full traceback has been saved in /tmp/sphinx-err-wLtHoF.log, if
>>> you want to report the issue to the developers.
>>>   Please also report this if it was a user error, so that a better error
>>> message can be provided next time.
>>>   A bug report can be filed in the tracker at
>>> <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
>>>   make[6]: *** [man] Error 1
>>>
>>> What did I do wrong ?
>>
>> You didn't do anything wrong that I can see. There was an internal
>> change to Docutils in revision 7942 that refactored some code, and a
>> side effect was to remove some attributes from the Inliner class (they
>> became locals instead). Your code depends on these class attributes.
>>
>> Try applying the attached patch and let us know how it works. The
>> missing class attributes will be present as instance attributes, which
>> should be close enough.
>>
>> Günter, I think the attached patch should also be rolled into a 0.13.2
>> bugfix release.
>>
>> David Goodger
>> <http://python.net/~goodger>
>>
>>
>>> On 01/01/2017 20:59, Guenter Milde wrote:
>>>> On 2016-12-31, Jean Baptiste Favre wrote:
>>>>> Hello,
>>>>> I'm facing an issue with Trafficserver documentation [1] which doesn't
>>>>> build with docutils 0.13.1
>>>>
>>>>> Error comes from a custom Inliner class which is defined in doc/conf.py
>>>>> of Trafficserver project(starting line 163).
>>>>> This allow to transform text like "(TS-XXXX)" in a link to
>>>>> trafficserver's Jira
>>>>
>>>>> This custom Inliner used to work with docutils 0.12. It fails on 0.13.1
>>>>> with following error:
>>>>
>>>>>   Exception occurred:
>>>>>     File "conf.py", line 185, in __init__
>>>>>       start_string_prefix=self.start_string_prefix,
>>>>>   AttributeError: Inliner instance has no attribute 'start_string_prefix'
>>>>
>>>>> Since docutils 0.13, start_string_prefix isn't statically defined. We
>>>>> have to call init_customiations for that.
>>>>
>>>> Yes, this changed with the implementation of the long expected feature
>>>>
>>>>   - Apply [ 103 ] Recognize inline markups without word boundaries.
>>>>
>>>> and the new configuration setting "character_level_inline_markup".
>>>>
>>>>
>>>>> First problem: one  must pass settings param when calling
>>>>> init_customizations, but I can't find any format or structure for it.
>>>>
>>>>> Second problem: I tried a workaround, creating a InlinerSettings class
>>>>> with minimal properties so that init_customizations could pass:
>>>>
>>>>>   class InlinerSettings:
>>>>>     character_level_inline_markup=None
>>>>>     pep_references=None
>>>>>     rfc_references=None
>>>>
>>>>> But, then, I faced another error:
>>>>
>>>> ...
>>>>
>>>> "settings" is a an object returned from the option parser (optparse module).
>>>>
>>>> There are others facing similar problems, e.g. Python distutils.
>>>> There, I learned the trick is to use
>>>>
>>>>   settings = frontend.OptionParser(components=(Parser,)).get_default_values()
>>>>
>>>>   -- https://hg.python.org/cpython/rev/db09d760b965
>>>>
>>>>
>>>>> Third problem: since the above tries didn't worked, I'd a look on custom
>>>>> directives and roles.
>>>>> But, it does not seems to be allowed to define a role with a custom regex.
>>>>> This would means I have to rewrite all "(TS-XXXX)" expression into ":TS:
>>>>> XXXX".
>>>>
>>>> This is the "minimal-invasive" approach, it would be work now but might save
>>>> issues later, as it depends less on Docutils internals.
>>>>
>>>>> Is there any way to achieve the migration in a compatible way with
>>>>> docutils 0.12 *and* without rewriting the docuemntation ?
>>>>
>>>> If the above example does not lead to a solution, you could also consider to
>>>> copy the definition of start_string_prefix ...
>>>> from states.py (importing utils.punctuation_chars first)
>>>>
>>>> Günter
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> Docutils-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/docutils-users
>
> Please use "Reply All" to reply to the list.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most 
engaging tech sites, SlashDot.org! http://sdm.link/slashdot

Index: docutils/parsers/rst/states.py
===================================================================
--- docutils/parsers/rst/states.py	(revision 8009)
+++ docutils/parsers/rst/states.py	(working copy)
@@ -479,11 +479,13 @@
                                  (punctuation_chars.closing_delimiters,
                                   punctuation_chars.delimiters,
                                   punctuation_chars.closers))
+        self.start_string_prefix = start_string_prefix
+        self.end_string_suffix = end_string_suffix
 
         args = locals().copy()
-        args.update(vars(self.__class__))
+        args.update(vars(Inliner))
 
-        parts = ('initial_inline', start_string_prefix, '',
+        parts = ('initial_inline', self.start_string_prefix, '',
            [('start', '', self.non_whitespace_after, # simple start-strings
              [r'\*\*',                # strong
               r'\*(?!\*)',            # emphasis but not strong
@@ -509,6 +511,7 @@
              )
             ]
            )
+        self.parts = parts
 
         self.patterns = Struct(
           initial=build_regexp(parts),