Re: memory leak in reportlab xmllib.FastXMLParser

On 17/12/2013 19:35, Mirko Dziadzka wrote:
> This solves the immediate problem. However, the problem reappear on higher levels.
>
> All code using ParaParser must call close() to  free the memory.
>
> But even reportlab internal code does not do this in every case.
> The program below reconstruct the memory leak on a higher level.
>
.........
Interestingly, with my fix in place the latest code doesn't fail with your 
example, I guess the error happens too early. I have now tried to close off the 
leak with changes to xmllib.py, paraparser.py & paragraph.py. I tested using the 
scripts below. The case for frags was important,  because originally we created 
a parser in Paragraph._setup and then never used it if frags was present.


####################################################
import  gc, time
from reportlab.lib import  xmllib

assert xmllib.sgmlop # check that we are using FastXMLParser

gc.set_debug(gc.DEBUG_LEAK)
while True:
    parser = xmllib.XMLParser(verbose=0)
    parser.close()
    gc.collect()
####################################################
import	gc
from reportlab.platypus.paragraph import Paragraph
from reportlab.lib.styles import ParagraphStyle

normalStyle = ParagraphStyle('normal')

# construct input which can not be decoded as utf-8
# this will throw an exception in the parser
while True:
	try:
		p = Paragraph('<para> </a>', normalStyle)
	except Exception, e:
		# will complain about invalid encoding ....
		# NOTE that I have NO WAY of cleaning up the memory here ...
		pass
	gc.collect()
####################################################
import  gc
from reportlab.platypus.paragraph import Paragraph
from reportlab.lib.styles import ParagraphStyle

normalStyle = ParagraphStyle('normal')

# construct input which can not be decoded as utf-8
# this will throw an exception in the parser
while True:
     try:
         p = Paragraph('<para> </a>', normalStyle, frags=[(1,'aaa')])
     except Exception, e:
         # will complain about invalid encoding ....
         # NOTE that I have NO WAY of cleaning up the memory here ...
         pass
     gc.collect()
####################################################
-- 
Robin Becker