Re: [PATCH] switch --update to use the sqlite dbs instead of the xml files. Should massively impact memory footprint and hopefully only marginally impact performance.

On Wed, 2011-09-07 at 12:18 -0400, Dennis Gregorovic wrote:
> On Fri, 2011-07-15 at 17:53 -0400, Seth Vidal wrote:
> > -    def getNodes(self, relpath):
> > -        """Return base, filelist, and other nodes for file, if they exist
> >  
> > -        Returns a tuple of nodes, or None if not found
> > +    def getNodes(self, relpath):
> > +        """return a package object based on relative path of pkg
> >          """
> > -        bnode = self.basenodes.get(relpath,None)
> > -        if bnode is None:
> > -            return None
> > -        pkgid = self.pkg_ids.get(relpath,None)
> > -        if pkgid is None:
> > -            print _("No pkgid found for: %s") % relpath
> > -            return None
> > -        fnode = self.filesnodes.get(pkgid,None)
> > -        if fnode is None:
> > -            return None
> > -        onode = self.othernodes.get(pkgid,None)
> > -        if onode is None:
> > -            return None
> > -        return bnode, fnode, onode
> > -
> > -    def freeNodes(self,relpath):
> > -        #causing problems
> > -        """Free up nodes corresponding to file, if possible"""
> > -        bnode = self.basenodes.get(relpath,None)
> > -        if bnode is None:
> > -            print "Missing node for %s" % relpath
> > -            return
> > -        bnode.unlinkNode()
> > -        bnode.freeNode()
> > -        del self.basenodes[relpath]
> > -        pkgid = self.pkg_ids.get(relpath,None)
> > -        if pkgid is None:
> > -            print _("No pkgid found for: %s") % relpath
> > +        if relpath in self.pkg_tups_by_path:
> > +            pkgtup = self.pkg_tups_by_path[relpath]
> > +            return self._repo.sack.searchPkgTuple(pkgtup)[0]
> > +        else:
> > +            print _("No pkg found for: %s") % relpath
> >              return None
> 
> Seth,
> 
> Thank you very much for this patch.  I've locally backported it to RHEL
> 6 and the memory savings are tremendous.  However, I just noticed one
> small issue.  Before, the getNodes call would only print "No pkg found
> for: %" when a package was found in the basenodes cache but not the
> pkg_ids cache.  That was an edge condition that never actually happened
> as far as I can tell, so the net effect was that "No pkg found" did not
> get printed.  After the code change, "No pkg found" is now printed every
> time getNodes is called with a package that isn't cached.  

Yah - I think someone else noticed that. Dgilmore, I think.

I was a bit torn about it - since it is quasi-useful to know which ones
went walk-about and which ones did not. However, it isn't helpful for
the general case of 'yes, I know I removed pkgs from the repo, just
rebuild it'.

I suspect it is really only useful when debugging something ridiculous.

Anyone object to me just pulling it?
-sv