Re: Package version conflicts

You put a some stuff in the db that would be really useful later, but
isn't required now. Cool - Current's upper layers could "grow" into the
low-level db that supports it.

However, dependancy resolution is a REQUIREMENT. Even Current's shelve
implementation doesn't do it all - we ignore conflicts and obsoletes
right now (I think I know what to do for obsoletes, but what to do with
or about conflicts is still an open discussion).

Basic dependancies (which I define as "whats in Current right now") come
in three flavors:
   file deps
   "name" deps
   provide deps

   File deps are easy - package X contains some file /usr/bin/foo
   Name deps are basically just the names of packages. If your    
      tree/channel contains the strace rpm, then "strace" will be here.
   provides are a little trickier - they _can_ be versioned, but aren't 
      always. Provides are almost always just libraries. 

Right now, Current ignores any version stuff. (it didn't look like the
client wanted it anyway). I put it in the DB because I finally figured
out how it was stored in the RPM, and wanted to "document" that
knowledge.

Any DB layer has a new rule "12": it must provide at least as much
dependancy data to Current as the existing shelve implementation.

The simplest way to do this that I can think of is 3 tables per channel,
with the needed data. However, one reason Current was written with
shelves is that I am not a DB programmer of any skill. 

Even if you don't grok all of it channel.py:getDependancy() demonstrates
how the data is used.


On Tue, 2002-01-29 at 22:47, Todd Nemanich wrote:
> Ok, here goes then. This is what I have so far.
> 
> There are 6 main entities in the schema so far 
> [Listed as Entity(attribute,attribute,...)]:
> - Package files(package_file,spec): package_file 
> can be either the rpm package itself, or text 
> entry of where it resides on the filesystem. Same 
> goes for spec, although spec can be NULL. See 
> notes below about this.
> 
> - Checksums(file,checksum,config): This contains 
> the checksums for the individual files within the 
> rpm. File is the full path to the file, checksum 
> is the checksum from the rpm package, config is a 
> boolean. More can probably be done here, but I 
> primarily aimed this as a way to provide a 
> checksum server.
> 
> - Packager(name,email,gpg_key): This is to hold 
> information about the packager. Name is a text 
> field with the packager's name, email is a text 
> field with their email address, and gpg_key hold 
> their public gpg (or pgp) key for verifying 
> signatures.
> 
> -Package_layers(Name): This is analagous to 
> entries in the comps file. A package layer would 
> be something like X Windows. Name is a text field 
> containing the name.
> 
> - Trees or Channels(Name): This is analagous to a 
> channel. Name is a text field containing the name.
> 
> - 
> Packages(package_name,package_version,package_release,epoch,description): 
> This should describe the package in general. 
> Package_name, package_version, and package_release 
> are text fields. epoch is a int4 right now, 
> although postgres does not allow that to be 
> unsigned. Description is a text field analagous to 
> the summary header.
> 
> Beyond the entities, there are several 
> relationships between them. These do affect how 
> the tables are defined in the database [Listed as 
> Entity(cardinality) <relationship> 
> Entity(cardinality) [relation attributes] ]:
> 
> Package_files(1) <contains> Packages(1) : This is 
> the mapping between the package file and the 
> package entry. Can be done by FK'ing (foreign 
> keying) the package name,version,release, and 
> epoch to package files.
> 
> Checksums(N) <matches> Packages(1) : This maps the 
> file contents and the checksums to an rpm. Again, 
> FK the package info to checksums.
> 
> Packager(1) <built> Packages(N) : This maps the 
> package to the packager. FK the name and or email 
> address from packagers to packages (This will 
> prevent unsigned packages from being inserted. See 
> below.)
> 
> Package_layers(N) <in> Trees/channels(M) : This 
> indicates which package layers are held by which 
> channel. This is provided by a third table 
> (tree/channel_layers) which joins channel id's 
> with layer id's via FK's.
> 
> Package_layers(1) <has sublayer> Package_layers(N) 
> [Qualifier] : This allows a layer to contain 
> another layer, dependent upon the qualifier being 
> met. An example from a comps file would be  within 
> a web server layer ? Java { Tomcat } Where tomcat 
> is another group of packages. This is provided 
> through an additional table that matches 
> (layer,sublayer,qualifier) where layer and 
> sublayer are FK's to name in package_layers.
> 
> Package_layers(N) <contains> Packages(M) 
> [Qualifier] : This is analagous to the packages 
> within a package layer in the comps file. This is 
> provided through a table 
> layer_contents(package_name,package_version,package_release,epoch,layer,qualifier) 
> where all but qualifier are FK's to packages or 
> package_layers.
> 
> Packages(N) <installed on> Devices(M) [Install 
> date, auto-update] : Ok, devices is part of 
> another database I work with, and I am restricted 
> from releasing the code for it. The important 
> piece from it is the device_id (a sequence used to 
> PK the entity) which is used for this mapping. The 
> relationship is provided by a table install_log 
> that FK's the package naming info and the 
> device_id. The install date is a timestamp and 
> auto-update is a boolean (in case you don't want 
> this package updated by up2date on the machine for 
> some reason).
> 
> 
> Whew! So that is the bulk of it. The real weakness 
> here is the issue of defining dependencies. I have 
> not had a chance to work that in yet. When I was 
> last working on this, I was sort of stumped 
> because dependencies can be a package (with or 
> without version info), a file, or a generic 
> service (like webserver or smtpdaemon). This would 
> be some sort of relationship from packages to 
> packages. I'm not sure how to define it though.
> 	I have an old schema file with the create 
> statements to setup this DB. It breaks rules 5-9 
> though. The RPMS are held within the database as 
> large objects in the package_files table. This is 
> done through a piece of C code (yes, now I break 
> rule 2, since I don't know python). The C code 
> takes the file as a stream, pulls all the rpm 
> headers and propogates them through the database. 
> It doesn't do a signature check yet, but that 
> should be easy enough to add. The C code is loaded 
> into the database as rpm_insert, and is not 
> pleasant (Although I will share if anyone wants 
> it). It works with rpms built by rpm v3 and rpm 
> v4. I could probably mod this to just read the 
> headers from the FS and propogate them through the 
> database.
> 	Also, on the subject of the unsigned packages, I'm 
> not sure if this is desired or not. This could 
> easily be cut, but I would prefer to make sure my 
> packages are always signed by someone I trust. 
> Currently, the FK in packages would prevent a 
> package not signed by a trusted packager from 
> being inserted.
> 	Ok, that pretty much does it. I would like to hear 
> comments about things that are 
> missing/needed/desired (like arch should be an 
> attribute of packages, etc). Particularly I would 
> like to hear ideas about solving the dependencies 
> issue. Later.
> 
> 
> Toby D. Reeves wrote:
> 
> > Hello everyone,
> > 
> > About a week ago I had an email discussion with Hunter about implementing a SQL 
> > backend for current.  I've got the bulk of the database stuff done.  I am now 
> > integrating it into current. It is not quite ready for anyone to try.  
> > Hopefully in about a week, depending on my schedule.
> > 
> > Ground rules:
> > 
> > 1. PostgreSQL (thats what Hunter wanted)
> > 2. Pure python 1.5.2
> > 3. Multiple rpm arch's per channel
> > 4. Sane handling of multiple rpm version/release/epoch per channel
> > 5. Directory recursion
> > 6. RPM, SRPMS, and headers on file systems so can be served with apache
> > 7. Works correctly with stock Redhat NFS/FTP install trees 
> > 8. Works correctly with simple mirror of updates.redhat.com
> > 9. Works with any other directory structure you care to have.
> > 10. Can add or remove rpms from channel without having to rebuild the entire 
> > database.
> > 11. Will support future "non-anonymous" mode so "update -p" works.
> > 
> > 
> > Toby
> > 
> > 
> 
> 
> _______________________________________________
> Current-server mailing list
> [email protected]
> http://lists.dulug.duke.edu/mailman/listinfo/current-server
> 
> 
-- 
Hunter Matthews                          Unix / Network Administrator
Office: BioScience 145/244               Duke Univ. Biology Department
Key: F0F88438 / FFB5 34C0 B350 99A4 BB02  9779 A5DB 8B09 F0F8 8438
Never take candy from strangers. Especially on the internet.