Re: mergerepo weird behavior

On 08/09/2011 11:23 AM, seth vidal wrote:
> On Tue, 2011-08-09 at 10:56 -0400, Jon Wallace wrote:
>    
>> Hello,
>>
>> I'm seeing some weird mergerepo behavior I can't make sense of.  I have
>> three repositories, PRODA, PRODB, PRODC, and each has a single package
>> within it.  I'm trying to merge various combinations of these together.
>> The example below tries to merge PRODB and PRODC into PROD-BC.
>>
>>
>> #mergerepo --repo=file:///<path-to-repo>/PRODB
>> --repo=file:///<path-to-repo>/PRODC -o PROD-BC
>> Loaded plugins: refresh-packagekit
>> Not using downloaded repomd.xml because it is older than what we have:
>>     Current   : Tue Aug  9 09:36:40 2011
>>     Downloaded: Mon Aug  8 17:50:36 2011
>> 3/3 - hp-PRODC-1.0-1.x86_64
>> Saving Primary metadata
>> Saving file lists metadata
>> Saving other metadata
>>
>> The "current" timestamp of Aug 9 9:36 corresponds to PRODA.
>> The "downloaded" timestamp of Aug 8 17:50 corresponds to PRODB.
>> Though not listed, PRODC repomd.xml has a timestamp of Aug 8 17:54.
>>
>> The result is that I end up with a merge of PRODA and PRODC even though
>> I asked for PRODB and PRODC.
>>
>> Here is the merged group file.
>> #cat groups.xml
>> <?xml version="1.0" encoding="UTF-8"?>
>> <!DOCTYPE comps PUBLIC "-//Red Hat, Inc.//DTD Comps info//EN" "comps.dtd">
>> <comps>
>>
>> <group>
>> <id>PRODA</id>
>> <default>true</default>
>> <uservisible>true</uservisible>
>> <display_order>1024</display_order>
>> <name>PROD-A Test Group</name>
>> <description>This is PROD-A #1111.</description>
>> <packagelist>
>> <packagereq type="default">PRODA</packagereq>
>> </packagelist>
>> </group>
>> <group>
>> <id>PRODC</id>
>> <default>true</default>
>> <uservisible>true</uservisible>
>> <display_order>1024</display_order>
>> <name>PROD-C Test Group</name>
>> <description>This is PROD-C #1111.</description>
>> <packagelist>
>> <packagereq type="default">PRODC</packagereq>
>> </packagelist>
>> </group>
>> <category>
>> <id>producta</id>
>> <display_order>10</display_order>
>> <name>Product-A Test Software</name>
>> <description>Product-A Test Software - Hello</description>
>> <grouplist>
>> <groupid>PRODA</groupid>
>> </grouplist>
>> </category>
>>
>> <category>
>> <id>productc</id>
>> <display_order>10</display_order>
>> <name>Product-C Test Software</name>
>> <description>Product-C Test Software - Hello</description>
>> <grouplist>
>> <groupid>PRODC</groupid>
>> </grouplist>
>> </category>
>>
>> </comps>
>>
>>
>> What am I doing wrong, or what have I set up wrong with my repositories?
>> Jon
>>
>>      
>
> what ver of createrepo/mergerepo is this?
>
> Can you go into /var/tmp/ and look for a tmpdir with your username in it
> and clear it out? maybe something like  /var/tmp/yum-$username-blah
>
> -sv
>
>    


Sorry for the delay and length of this email, I had to rebuild my test 
environment but at least I've been able to duplicate the problem again.  
Also trying to provide as much info as possible.

OS = Fedora Core 11
createrepo = 0.9.7-7

I cleared /var/tmp/yum-<myname>-blah and confirmed that solves the 
problem, but only if I clear it prior to each time I run mergerepo.  I 
suppose I can do that but it seems a bit of a pain and should probably 
be documented if that's the fix.  Here's a synopsis of the problem 
again, and I've included the contents of some files.

REPOSITORY CONTENTS
./PRODA
./PRODA/repodata
./PRODA/repodata/repomd.xml
./PRODA/repodata/groups.xml.gz
./PRODA/repodata/other.xml.gz
./PRODA/repodata/groups.xml
./PRODA/repodata/primary.xml.gz
./PRODA/repodata/filelists.xml.gz
./PRODA/PRODA-1.0-1.x86_64.rpm
./PRODA/groups.xml
./PRODB
./PRODB/repodata
./PRODB/repodata/repomd.xml
./PRODB/repodata/groups.xml.gz
./PRODB/repodata/other.xml.gz
./PRODB/repodata/groups.xml
./PRODB/repodata/primary.xml.gz
./PRODB/repodata/filelists.xml.gz
./PRODB/PRODB-1.0-1.x86_64.rpm
./PRODB/groups.xml
./PRODC
./PRODC/repodata
./PRODC/repodata/repomd.xml
./PRODC/repodata/groups.xml.gz
./PRODC/repodata/other.xml.gz
./PRODC/repodata/groups.xml
./PRODC/repodata/primary.xml.gz
./PRODC/repodata/filelists.xml.gz
./PRODC/PRODC-1.0-1.x86_64.rpm
./PRODC/groups.xml

Note that each package contains only a single file trying to make the 
contents of the repository as simple as possible.

$ cat PRODA/groups.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE comps PUBLIC "-//Red Hat, Inc.//DTD Comps info//EN" "comps.dtd">
<comps>

<group>
<id>PRODA</id>
<default>true</default>
<uservisible>true</uservisible>
<display_order>1024</display_order>
<name>PROD-A Test Group</name>
<description>This is PROD-A #1111.</description>
<packagelist>
<packagereq type="default">hp-PRODA</packagereq>
</packagelist>
</group>

<category>
<id>producta</id>
<display_order>10</display_order>
<name>Product-A Test Software</name>
<description>Product-A Test Software Prototype Evaluation</description>
<grouplist>
<groupid>PRODA</groupid>
</grouplist>
</category>

</comps>


$ cat PRODB/groups.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE comps PUBLIC "-//Red Hat, Inc.//DTD Comps info//EN" "comps.dtd">
<comps>

<group>
<id>PRODB</id>
<default>true</default>
<uservisible>true</uservisible>
<display_order>1024</display_order>
<name>PROD-B Test Group</name>
<description>This is PROD-B #1111.</description>
<packagelist>
<packagereq type="default">hp-PRODB</packagereq>
</packagelist>
</group>

<category>
<id>productb</id>
<display_order>10</display_order>
<name>Product-B Test Software</name>
<description>Product-B Test Software Prototype Evaluation</description>
<grouplist>
<groupid>PRODB</groupid>
</grouplist>
</category>

</comps>


$ cat PRODC/groups.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE comps PUBLIC "-//Red Hat, Inc.//DTD Comps info//EN" "comps.dtd">
<comps>

<group>
<id>PRODC</id>
<default>true</default>
<uservisible>true</uservisible>
<display_order>1024</display_order>
<name>PROD-C Test Group</name>
<description>This is PROD-C #1111.</description>
<packagelist>
<packagereq type="default">hp-PRODC</packagereq>
</packagelist>
</group>

<category>
<id>productc</id>
<display_order>10</display_order>
<name>Product-C Test Software</name>
<description>Product-C Test Software Prototype Evaluation</description>
<grouplist>
<groupid>PRODC</groupid>
</grouplist>
</category>

</comps>




1) Start with a clean /var/tmp/yum-<myname>, then merge PRODA, PRODB, 
PRODC into PROD-ABC.  This completes as expected with no errors.

2) DO NOT clear /var/tmp/yum-<myname>, then merge PRODB and PRODC into 
PROD-BC, and I get the following results.

$ mergerepo --repo=file:///<absoute-path>/PRODB 
--repo=file:///<absolute-path>/PRODC -o PROD-BC
Loaded plugins: refresh-packagekit
Not using downloaded repomd.xml because it is older than what we have:
   Current   : Wed Aug 10 16:34:51 2011
   Downloaded: Wed Aug 10 16:14:11 2011
2/2 - hp-PRODC-1.0-1.x86_64
Saving Primary metadata
Saving file lists metadata
Saving other metadata

$ ls PROD-BC
total 4
drwxr-xr-x. 2 jon jon 4096 2011-08-10 16:41 repodata

$ ls PROD-BC/repodata
total 28
-rw-r--r--. 1 jon jon  374 2011-08-10 16:41 filelists.xml.gz
-rw-r--r--. 1 jon jon 1328 2011-08-10 16:41 groups.xml
-rw-r--r--. 1 jon jon  457 2011-08-10 16:41 groups.xml.gz
-rw-r--r--. 1 jon jon  344 2011-08-10 16:41 other.xml.gz
-rw-r--r--. 1 jon jon  824 2011-08-10 16:41 primary.xml.gz
-rw-r--r--. 1 jon jon 2091 2011-08-10 16:41 repomd.xml
-rw-r--r--. 1 jon jon  116 2011-08-10 16:41 updateinfo.xml.gz


HERE IS THE MERGED GROUPS FILE.
Note that even though I merged PRODB and PRODC, the groups file has an 
entry for PRODA in both the group and category tags.

$ cat groups.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE comps PUBLIC "-//Red Hat, Inc.//DTD Comps info//EN" "comps.dtd">
<comps>

<group>
<id>PRODA</id>
<default>true</default>
<uservisible>true</uservisible>
<display_order>1024</display_order>
<name>PROD-A Test Group</name>
<description>This is PROD-A #1111.</description>
<packagelist>
<packagereq type="default">hp-PRODA</packagereq>
</packagelist>
</group>
<group>
<id>PRODC</id>
<default>true</default>
<uservisible>true</uservisible>
<display_order>1024</display_order>
<name>PROD-C Test Group</name>
<description>This is PROD-C #1111.</description>
<packagelist>
<packagereq type="default">hp-PRODC</packagereq>
</packagelist>
</group>
<category>
<id>producta</id>
<display_order>10</display_order>
<name>Product-A Test Software</name>
<description>Product-A Test Software Prototype Evaluation</description>
<grouplist>
<groupid>PRODA</groupid>
</grouplist>
</category>

<category>
<id>productc</id>
<display_order>10</display_order>
<name>Product-C Test Software</name>
<description>Product-C Test Software Prototype Evaluation</description>
<grouplist>
<groupid>PRODC</groupid>
</grouplist>
</category>

</comps>



HERE IS THE MERGED filelists.xml FILE.
Note it contains the PRODA package even though I merged PRODB and PRODC.

$ cat filelists.xml.gz | gunzip
<?xml version="1.0" encoding="UTF-8"?>
<filelists xmlns="http://linux.duke.edu/metadata/filelists" packages="2">
<package 
pkgid="85fca2cee6628eea8f6aa67001606e3585380b1ff6ba8dce2dfef57e38d1d6d1" 
name="hp-PRODA" arch="x86_64">
<version epoch="0" ver="1.0" rel="1"/>
<file>/opt/HP/test1/fileA-1</file>
</package>

<package 
pkgid="b681e126c401780913c2878c253f8e513520f23194117e77c82bb5d4ecbeaa3c" 
name="hp-PRODC" arch="x86_64">
<version epoch="0" ver="1.0" rel="1"/>
<file>/opt/HP/test1/fileC-1</file>
</package>



HERE IS THE MERGED primary.xml FILE.
Note that the <location> tag has a reference to the PRODB repository but 
lists the package as PRODA.  The only thing I have changed here is the 
path to the repositories with <absolute-path> to protect the 
innocent...(me).

$ cat primary.xml.gz | gunzip
<?xml version="1.0" encoding="UTF-8"?>
<metadata xmlns="http://linux.duke.edu/metadata/common" 
xmlns:rpm="http://linux.duke.edu/metadata/rpm" packages="2">
<package type="rpm">
<name>hp-PRODA</name>
<arch>x86_64</arch>
<version epoch="0" ver="1.0" rel="1"/>
<checksum type="sha256" 
pkgid="YES">85fca2cee6628eea8f6aa67001606e3585380b1ff6ba8dce2dfef57e38d1d6d1</checksum>
<summary>HP PROD-A Test RPM x86_64</summary>
<description>This is PROD-A test RPM ID# 1296569852</description>
<packager></packager>
<url></url>
<time file="1313007190" build="1313006939"/>
<size package="2023" installed="2023" archive="316"/>
<location xml:base="file:///<absolute-path>/PRODB" 
href="hp-PRODA-1.0-1.x86_64.rpm"/>
<format>
<rpm:license>GPLv2+</rpm:license>
<rpm:vendor/>
<rpm:group>HPTEST</rpm:group>
<rpm:buildhost>FC11</rpm:buildhost>
<rpm:sourcerpm>hp-PRODA-1.0-1.src.rpm</rpm:sourcerpm>
<rpm:header-range start="280" end="1856"/>
<rpm:provides>
<rpm:entry name="hp-PRODA(x86-64)" flags="EQ" epoch="0" ver="1.0" rel="1"/>
<rpm:entry name="hp-PRODA" flags="EQ" epoch="0" ver="1.0" rel="1"/>
</rpm:provides>
<rpm:requires>
<rpm:entry name="/bin/sh" pre="0"/>
</rpm:requires>
</format>
</package>
<package type="rpm">
<name>hp-PRODC</name>
<arch>x86_64</arch>
<version epoch="0" ver="1.0" rel="1"/>
<checksum type="sha256" 
pkgid="YES">b681e126c401780913c2878c253f8e513520f23194117e77c82bb5d4ecbeaa3c</checksum>
<summary>HP PROD-C Test RPM x86_64</summary>
<description>This is PROD-C test RPM ID# 1296569852</description>
<packager></packager>
<url></url>
<time file="1313007214" build="1313006978"/>
<size package="2020" installed="2020" archive="316"/>
<location xml:base="file:///<absolute-path>/PRODC" 
href="hp-PRODC-1.0-1.x86_64.rpm"/>
<format>
<rpm:license>GPLv2+</rpm:license>
<rpm:vendor/>
<rpm:group>HPTEST</rpm:group>
<rpm:buildhost>FC11</rpm:buildhost>
<rpm:sourcerpm>hp-PRODC-1.0-1.src.rpm</rpm:sourcerpm>
<rpm:header-range start="280" end="1856"/>
<rpm:provides>
<rpm:entry name="hp-PRODC(x86-64)" flags="EQ" epoch="0" ver="1.0" rel="1"/>
<rpm:entry name="hp-PRODC" flags="EQ" epoch="0" ver="1.0" rel="1"/>
</rpm:provides>
<rpm:requires>
<rpm:entry name="/bin/sh" pre="0"/>
</rpm:requires>
</format>
</package>
</metadata>