Re: [WBEL-devel] Choosing a RHEL rebuild project

David Cox <[email protected]> Wed, 21 Jan 2004 17:53:12 -0500

Newsgroups	gmane.linux.whitebox.devel
Organization	NASA Langley Research Center
Message-ID	<[email protected]>

On Wed, 2004-01-21 at 16:09, Henk van Lingen wrote:

>   Maybe someone has some idea's about how to find and judge the differences?
>   Since this week, I also have a dell 2650 with RHEL 3 AS up and running
>   as a reference (besides being a postgresql server).

I posted this awhile back, and David Parsley has something similar in
tao, but attached is a script I use to evaluate rebuilds.  It compares
rebuilds file-by-file on completeness (extra or missing files),
comparison of dynamic links and file size.  It does this by extracting
files from each rpm one-at-a-time in a scratch directory, so there is no
need to install a distribution or run it on a particular distribution.
It won't mess with your rpm database, and doesn't need to be run as
root.

I've run this on the whitebox, tao and centos rpms.  Tao compares a
little better than Whitebox particularly on file size. Centos had many
link problems, but it wasn't the latest build.  I'm pretty proud of my
rebuilds, but I cheated a bit and took 275 binary packages straight out
of redhat-9, as these are identical to the binaries in RHEL.  These
common binaries highlight the fact that RHEL is not a self-hosting OS
and may not even have a unique buildhost.  So a self-hosting rebuild is
never going to hit the mark exactly.

Just give this a go on one of the distro's rpms and your 3AS RPM's.  All
you should need to do is edit the paths at the top of the file and maybe
adjust the size-error threshold.

dave

-- 
--
   David E. Cox        [email protected]      (757) 864-6658
--

testrebuild.pl (text/x-perl, 3.2 KB)

#!/usr/bin/perl -w

# Define Redhat and Comparison (WhiteBox) RPM Dirs.
$RH_RPMS = "/usr/local/rhel-3ws/RPMS/";
$WB_RPMS = "/usr/local/whitebox/RPMS/";

# Size difference to trigger error, .1 = plus/minus 10%
$SizeError=.10;

# Working directories
# Careful where these point, they get a "rm -rf" cleaning!
$RHTMP="/tmp/RH-$$";
$WBTMP="/tmp/WB-$$";

opendir(DIR,$RH_RPMS);
@allrpms= grep (/rpm$/, readdir(DIR) );
closedir(DIR);

foreach $rpm (sort @allrpms){
    if (! (-e "$WB_RPMS/$rpm") ) {
	print " No equivalent rebuild for $rpm \t Skipping\n";
	next;
    }
    # Initialize
    %allfiles=(); %RHhash=(); %WBhash=();
    $fileproblems=0; $libproblems=0;
    system("rm -rf $RHTMP;mkdir $RHTMP");
    system("rm -rf $WBTMP;mkdir $WBTMP");
    # Extract RPMS
    system("cd $RHTMP/; rpm2cpio $RH_RPMS/$rpm | cpio -id --no-absolute-filenames --quiet");
    system("cd $WBTMP/; rpm2cpio $WB_RPMS/$rpm | cpio -id --no-absolute-filenames --quiet");

    # Parse name/size info
    @RHnamesize=`cd $RHTMP/; find ./ -printf "%p:_:%s\n"`;
    @WBnamesize=`cd $WBTMP/; find ./ -printf "%p:_:%s\n"`;
    for ($i=0;$i<@RHnamesize;$i++){
	($file,$size)=split(/:_:/,$RHnamesize[$i]);
	$RHhash{"$file"}=$size;
	$allfiles{"$file"}=1;
    }
    for ($i=0;$i<@WBnamesize;$i++){
	($file,$size)=split(/:_:/,$WBnamesize[$i]);
	$WBhash{"$file"}=$size;
	$allfiles{"$file"}=1;
    }

    
    # Print and tag file errors if appropraite
    printf ("%-50s\t", $rpm);
    foreach $file (keys %allfiles){
	if (!(exists $RHhash{"$file"})){
	    print "\n\tPackage contains extra file:  $file";
	    $fileproblems=1;
	}
	elsif (!(exists $WBhash{"$file"})){
	    print "\n\tPackage is missing file: $file";
	    $fileproblems=1;
	}
	else {
	    if ($RHhash{"$file"}>0){
		$ratio=($WBhash{"$file"})/($RHhash{"$file"})}
	    else{
		$ratio=1-$WBhash{"$file"};  # Negative ratios indicate filesize vs. zero in RedHat
	    }
	    if (($ratio-1)*($ratio-1)>($SizeError*$SizeError)) {
		printf("\n\tSize Ratio: %3.2f on file:%s",$ratio,$file);
		$fileproblems=1;
	    };
	}
    }

    
    # Select executable files
    @RHexec=`cd $RHTMP/; find ./ -perm +111 -type f -print`; 

    # Check library dependencies
    foreach $executable (@RHexec) {
	chop $executable;
	#  Move on if no equivalent binary exists in rebuild
	if (! (-e "$WBTMP/$executable") ) {next;}

	# Get shared libs
	@RHlibs=`ldd $RHTMP/$executable`;
	@WBlibs=`ldd $WBTMP/$executable`;

        # Clean up ldd output
	$RHlibline=join('',(sort @RHlibs)); 
	$WBlibline=join('',(sort @WBlibs)); 
	$RHlibline=~ s/=>(.*)(\n)/\n/g; $RHlibline=~ s/(\s+)/ /g; 
	$WBlibline=~ s/=>(.*)(\n)/\n/g; $WBlibline=~ s/(\s+)/ /g;
	@RHlibs=split(/ /,$RHlibline);
	@WBlibs=split(/ /,$WBlibline);

        # Print and tag library errors
	foreach $lib (@RHlibs){
	    $searchpat=$lib;
	    $searchpat =~ s/\+/\\\+/g;
	    $searchpat =~ s/\./\\\./g;
	    unless ($WBlibline =~ (/$searchpat/)){
		print "\n\tMissing link in $executable:\t$lib";$libproblems=1;
	    }
	}
	    
	foreach $lib (@WBlibs){
	    $searchpat=$lib;
	    $searchpat =~ s/\+/\\\+/g;
	    $searchpat =~ s/\./\\\./g;
	    unless ($RHlibline =~ (/$searchpat/)){
		print "\n\tExtra link in $executable:\t$lib";$libproblems=1;
	    }
	}
    }
    if ($fileproblems || $libproblems) { print "\n\n";} else {print "MATCH\n";} 
}