Re: createrepo testing please.

On Tue, Jan 13, 2004 at 10:28:56PM -0500, seth vidal wrote:
> On Tue, 2004-01-13 at 17:42, Daniel Veillard wrote:
> > On Tue, Jan 13, 2004 at 04:55:35PM -0500, seth vidal wrote:
> > > tell me anything that breaks. 
> > 
> >   not really "breaking" but there are some usability improvement
> > which may or may not be possible:
> >    - memory usage (150 MB indexing FC1 i386/os)
> 
> fixed - by doing dumping it to a string and writing each package node
> out then node.unlinkNode() node.freeNode() I've gotten rid of all the
> built up memory. it now maxes out at 7MB instead of whatever the size of
> the xml doc is.

  Cool :-)

> >    - reusing the existing XML as a cache on subsequent runs
> 
> this will take more time. But I'd like to do it.

  Suggestions:
    - scan initially the primary.xml.gz to extract just the pathnames
    - do the file lookup (xmlReader interface sounds just fine).
    - build a list of added packages and removed packages
    - scan the added packages, dump the descriptions for those
    - then rerun the reader on primary.xml.gz, expand the nodes that
      are kept unmodified and use the same dump routine you use now
      to output them (see http://xmlsoft.org/xmlreader.html#Mixing
      for an example of the reader and expansion code).

I think that way the impact will be minimal, both in terms of speed,
memory usage and code changes for reindexing.

Daniel

-- 
Daniel Veillard      | Red Hat Network https://rhn.redhat.com/
[email protected]  | libxml GNOME XML XSLT toolkit  http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/