Re: Re: portage performance
Jesse Guardiani <[email protected]>
| Newsgroups | gmane.linux.gentoo.performance |
|---|---|
| Organization | WingNET |
| Message-ID | <[email protected]> |
Brian Harring wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > >> The basic problem in searching is actually that it isn't implemented >> smartly >> in current portage. I have working (emerge -s like) code that is >> blazingly >> fast as it does not actually open all ebuilds. > Searching works off of the cache for the most part, if a cache entry is > stale, it's updated (eg the ebuild is opened and srced). > Unless you're not checking the cache and updating it as you proceed, > you're implementation ought to suffer the same limitation. > >> Doing description searching is >> impossible to do fast without some kind of cache. I don't think >> creating a >> reliable cache for that is going to be a priority, > There are 2 things that need to be done (in my books at least) to step > up the speed of a description search- > A) sql based cache backend, whether sqlite or mysql. If it comes to this, I definately vote for sqlite. It's much faster than mysql for embedded apps, and you don't have to worry about whether or not the server is running. > Either that, or > extend the flat cache to store the descriptions in a central index. I think this is a good idea. AFAIK, this is what FreeBSD does. Perhaps tinycdb could be used? (OCaml has a CDB module available too. You can get it here: http://bleu.west.spy.net/~dustin/projects/ocaml/index.xtp (BSD licence)) > B) alter the search description alg so that instead of stepping through > each entry getting the description, we just state "give me all packages > that have a description matching blar", and leave it up to the backend > to decide what is the most efficient way to search. With flat cache, > we'd still have to go file by file; w/ a sql variant, it could take > advantage of the appropriate syntax. I personally prefer well designed hash table schemes. This is part of a base operating system, so it needs to be extremely efficient. IMO, SQL databases tend to encourage laziness. This is not to say that they don't have their place, but I don't think they belong in a package management system. Still, it's a novel idea, I think. And offers a great deal of flexibility. -- Jesse Guardiani, Systems Administrator WingNET Internet Services, P.O. Box 2605 // Cleveland, TN 37320-2605 423-559-LINK (v) 423-559-5145 (f) http://www.wingnet.net -- [email protected] mailing list