Re: gzimg hashes

Brook Milligan <[email protected]> Tue, 7 Jul 2026 10:52:43 -0600
Newsgroups	gmane.os.netbsd.current
Message-ID	<[email protected]>
> On Jul 6, 2026, at 10:49, Brook Milligan <[email protected]> =
wrote:
>=20
>=20
>> On Jul 5, 2026, at 20:12, David H. Gutteridge <[email protected]> =
wrote:
>>=20
>> On Sun, 5 Jul 2026 at 17:42:17 -0600, Brook Milligan wrote:
>>> I notice that the hashes generated for release files in the gzimg
>>> directory cover the uncompressed files, not the compressed files
>>> actually distributed.  This seems to be a mistake.
>>>=20
>>> The fix is clearly to reorder the hash generation relative to the
>>> compression, but I=E2=80=99m checking here to make sure I=E2=80=99m =
not missing
>>> something.
>>>=20
>>> Cheers,
>>> Brook
>>=20
>> Yes, that's one of the issues raised in PR 59195 (and its duplicate,
>> PR 60302), e.g.,
>> =
https://nam10.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Fmail-i=
ndex.netbsd.org%2Fnetbsd-bugs%2F2025%2F04%2F03%2Fmsg088095.html&data=3D05%=
7C02%7Cbrook%40biology.nmsu.edu%7Cce313e1520f249b728b708dedb04114e%7Ca3ec8=
7a89fb84158ba8ff11bace1ebaa%7C1%7C0%7C639189007694159583%7CUnknown%7CTWFpb=
GZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjo=
iTWFpbCIsIldUIjoyfQ%3D%3D%7C60000%7C%7C%7C&sdata=3D%2Bv5UsD4R9GqmbVitcRxAG=
%2F81hAsh4KAIIb0ILBfSbhc%3D&reserved=3D0
>=20
> OK, thanks.  I have a patch, see below, that solves most of this. =20

Here is (I believe) a better patch + the referenced mkhashes script.  It =
does the following:

- reverts the commit to src/distrib/utils/embedded/mkimage (rev 1.91) so =
mkimage is no longer responsible for hashes

- creates the script src/distrib/utils/embedded/mkimage, which makes a =
standard set of hashes.  The boilerplate is based upon mkimage, so I =
hope that is OK.

- uses mkhashes to generate hashes for all (that I know of) users, i.e., =
in src/etc/etc.*/Makefile.inc

Note that only a single hash for any file will be retained and multiple =
hashes will accumulate in the files.

I think this solves the issues mentioned in PR 59195 and PR 60302.

Comments are welcome.

Thanks a lot.

Cheers,
Brook


Index: distrib/utils/embedded/mkimage
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: /cvsroot/src/distrib/utils/embedded/mkimage,v
retrieving revision 1.97
diff -u -r1.97 mkimage
--- distrib/utils/embedded/mkimage	22 May 2026 14:44:16 -0000	=
1.97
+++ distrib/utils/embedded/mkimage	7 Jul 2026 16:36:35 -0000
@@ -425,8 +425,4 @@
 	image=3D"${image}.gz"
 fi
=20
-cd "${IMAGEDIR}"
-${CKSUM} -a MD5 "$(basename "${image}")" > MD5
-${CKSUM} -a SHA512 "$(basename "${image}")" > SHA512
-
 echo ${bar} Image is ${image} ${bar}
Index: etc/etc.evbarm/Makefile.inc
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: /cvsroot/src/etc/etc.evbarm/Makefile.inc,v
retrieving revision 1.138
diff -u -r1.138 Makefile.inc
--- etc/etc.evbarm/Makefile.inc	20 Feb 2026 07:20:28 -0000	1.138
+++ etc/etc.evbarm/Makefile.inc	7 Jul 2026 16:36:35 -0000
@@ -3,6 +3,7 @@
 #	etc.evbarm/Makefile.inc -- evbarm-specific etc Makefile targets
 #
=20
+MKHASHES=3D${NETBSDSRCDIR}/distrib/utils/embedded/mkhashes
 MKIMAGE=3D ${NETBSDSRCDIR}/distrib/utils/embedded/mkimage
=20
 # If you change the list of distributed kernels, don't forget
@@ -96,8 +97,6 @@
 	MACHINE=3D${MACHINE} \
 	MKDTB=3D${MKDTB} \
 	MKPOSTFIX=3D${MKPOSTFIX} \
-	CKSUM=3D${CKSUM} \
-	IMAGEDIR=3D${IMAGE.dir} \
 	${HOST_SH} ${MKIMAGE} -h ${.TARGET:S/smp_//} -D ${DESTDIR} \
 	    -S ${NETBSDSRCDIR} -B ${IMAGEENDIAN} ${MKI_OPTS.${.TARGET}} =
\
 	    ${IMAGE.dir}/${.TARGET:S/smp_//}.img
@@ -107,10 +106,12 @@
 	@${TOOL_INSTALLBOOT} -m ${MACHINE} -o board=3D${f} \
 			${IMAGE.dir}/${.TARGET:S/smp_//}-${f}.img \
 		&& ${TOOL_GZIP} -f =
${IMAGE.dir}/${.TARGET:S/smp_//}-${f}.img \
+		&& ${MKHASHES} -c ${CKSUM} =
${IMAGE.dir}/${.TARGET:S/smp_//}-${f}.img.gz \
 		|| rm ${IMAGE.dir}/${.TARGET:S/smp_//}-${f}.img
 .endfor
 	@echo "=3D=3D=3D> Compressing image =
${IMAGE.dir}/${.TARGET:S/smp_//}.img"
 	${TOOL_GZIP} -f ${GZIP_N_FLAG} -9 =
${IMAGE.dir}/${.TARGET:S/smp_//}.img
+	${MKHASHES} -c ${CKSUM} ${IMAGE.dir}/${.TARGET:S/smp_//}.img.gz=20=

=20
 ARCHES.earmv4=3D		armv4
 ARCHES.earmv4eb=3D	armv4
Index: etc/etc.evbmips/Makefile.inc
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: /cvsroot/src/etc/etc.evbmips/Makefile.inc,v
retrieving revision 1.32
diff -u -r1.32 Makefile.inc
--- etc/etc.evbmips/Makefile.inc	22 Mar 2025 09:25:14 -0000	=
1.32
+++ etc/etc.evbmips/Makefile.inc	7 Jul 2026 16:36:35 -0000
@@ -52,6 +52,9 @@
 IMAGEENDIAN=3D	le
 .endif
=20
+# Support for mkhashes
+MKHASHES=3D 	${NETBSDSRCDIR}/distrib/utils/embedded/mkhashes
+
 __mkimage: .USE
 	TOOL_MAKE=3D${MAKE} \
 	TOOL_MAKEFS=3D${TOOL_MAKEFS} \
@@ -70,11 +73,10 @@
 	MACHINE=3D${MACHINE} \
 	MKDTB=3D${MKDTB} \
 	MKPOSTFIX=3D${MKPOSTFIX} \
-	CKSUM=3D${CKSUM} \
-	IMAGEDIR=3D${IMAGE.dir} \
 	${HOST_SH} ${MKIMAGE} -h ${.TARGET:S/smp_//} -D ${DESTDIR} \
             -S ${NETBSDSRCDIR} -B ${IMAGEENDIAN} ${MKI_OPTS.${.TARGET}} =
\
             ${IMAGE.dir}/${.TARGET:S/smp_//}.img.gz
+	${MKHASHES} -c ${CKSUM} ${IMAGE.dir}/${.TARGET:S/smp_//}.img.gz
=20
 .if !empty(MACHINE_ARCH:Mmips*64eb)
 smp_octeon: __mkimage
Index: etc/etc.evbppc/Makefile.inc
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: /cvsroot/src/etc/etc.evbppc/Makefile.inc,v
retrieving revision 1.22
diff -u -r1.22 Makefile.inc
--- etc/etc.evbppc/Makefile.inc	10 Jan 2026 06:56:58 -0000	1.22
+++ etc/etc.evbppc/Makefile.inc	7 Jul 2026 16:36:35 -0000
@@ -39,6 +39,9 @@
 IMAGE.kern=3D     ${IMAGE.rel}/binary/kernel
 IMAGEENDIAN=3D	be
=20
+# Support for mkhashes
+MKHASHES=3D 	${NETBSDSRCDIR}/distrib/utils/embedded/mkhashes
+
 .if ${MACHINE_ARCH} =3D=3D "powerpc"
 __mkimage: .USE
 	TOOL_MAKE=3D${MAKE} \
@@ -58,11 +61,10 @@
 	MACHINE=3D${MACHINE} \
 	MKDTB=3D${MKDTB} \
 	MKPOSTFIX=3D${MKPOSTFIX} \
-	CKSUM=3D${CKSUM} \
-	IMAGEDIR=3D${IMAGE.dir} \
 	${HOST_SH} -x ${MKIMAGE} -h ${.TARGET:S/smp_//} -D ${DESTDIR} \
             -S ${NETBSDSRCDIR} -B ${IMAGEENDIAN} ${MKI_OPTS.${.TARGET}} =
\
             ${IMAGE.dir}/${.TARGET:S/smp_//}.img.gz
+	${MKHASHES} -c ${CKSUM} ${IMAGE.dir}/${.TARGET:S/smp_//}.img.gz
=20
 smp_nintendo: __mkimage
 	cd ${KERNSRCDIR}/arch/evbppc/stand/wii && ${MAKE} release
Index: etc/etc.riscv/Makefile.inc
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: /cvsroot/src/etc/etc.riscv/Makefile.inc,v
retrieving revision 1.7
diff -u -r1.7 Makefile.inc
--- etc/etc.riscv/Makefile.inc	22 Mar 2025 09:25:14 -0000	1.7
+++ etc/etc.riscv/Makefile.inc	7 Jul 2026 16:36:35 -0000
@@ -15,6 +15,9 @@
 IMAGE.dir=3D      ${IMAGE.rel}/binary/gzimg
 IMAGE.kern=3D     ${IMAGE.rel}/binary/kernel
=20
+# Support for mkhashes
+MKHASHES=3D 	${NETBSDSRCDIR}/distrib/utils/embedded/mkhashes
+
 #	TOOL_MKUBOOTIMAGE=3D${TOOL_MKUBOOTIMAGE} \
=20
 __mkimage: .USE
@@ -34,12 +37,10 @@
 	MACHINE=3D${MACHINE} \
 	MKDTB=3D${MKDTB} \
 	MKPOSTFIX=3D${MKPOSTFIX} \
-	CKSUM=3D${CKSUM} \
-	IMAGEDIR=3D${IMAGE.dir} \
 	${HOST_SH} ${MKIMAGE} -h ${.TARGET:S/smp_//} -D ${DESTDIR} \
             -S ${NETBSDSRCDIR} ${MKI_OPTS.${.TARGET}} \
             ${IMAGE.dir}/${.TARGET:S/smp_//}.img.gz
-
+	${MKHASHES} -c ${CKSUM} ${IMAGE.dir}/${.TARGET:S/smp_//}.img.gz
=20
 .if ${MACHINE_ARCH} =3D=3D "riscv64"
 .if ${HAVE_UEFI} =3D=3D "yes"



#!/bin/sh
# $NetBSD$
#
# Copyright (c) 2026 The NetBSD Foundation, Inc.
# All rights reserved.
#
# This code is derived from software contributed to The NetBSD =
Foundation
# by Brook Milligan.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# 1. Redistributions of source code must retain the above copyright
#    notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
#    notice, this list of conditions and the following disclaimer in the
#    documentation and/or other materials provided with the =
distribution.
# 3. Neither the name of The NetBSD Foundation nor the names of its
#    contributors may be used to endorse or promote products derived
#    from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND =
CONTRIBUTORS
# ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT =
LIMITED
# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A =
PARTICULAR
# PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE FOUNDATION OR =
CONTRIBUTORS
# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR =
BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER =
IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR =
OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF =
THE
# POSSIBILITY OF SUCH DAMAGE.
#

#
# Makes a set of hashes for a file.
#

set -e

PROG=3D"$(basename "$0")"

cksum=3D${CKSUM:-cksum}
hashes=3D${HASHES:-"MD5 SHA512"}
imagedir=3D${IMAGEDIR}

bar=3D"=3D=3D=3D"

tmp=3D"$(mktemp -d "${TMPDIR:-/tmp}/$PROG.XXXXXX")"

trap "cleanup" 0 1 2 3 15

cleanup() {
	case "$tmp" in
	"${TMPDIR:-/tmp}/$PROG."*)	rm -fr "$tmp";;
	esac
}

usage() {
	cat << EOF 1>&2
Usage: $PROG [-H <hashes>] [-c <cksum>] [-d <imagedir>] <image>

-H	Hash algorithms [${hashes}]
-c	Checksum program [${cksum}]
-d	Image directory [dirname \${image}]
EOF
	exit 1
}

OPTS=3D"H:c:d:"
while getopts "$OPTS" f
do
	case $f in
	H)	hashes=3D"$OPTARG";;
	c)	cksum=3D"$OPTARG";;
	d)	imagedir=3D"$OPTARG";;
	*)	usage;;
	esac
done

shift $(( $OPTIND - 1 ))
if [ -n "$1" ]; then
	# take the next argument as being the image name
	image=3D"$1"
	shift
fi

if [ -z "${image}" ]; then
	echo "${PROG}: image undefined"
	usage
fi

if [ -z "${imagedir}" ]; then
	imagedir=3D"$(dirname "${image}")"
fi

image=3D"$(basename "${image}")"

cd "${imagedir}"

if [ ! -f ${image} ]; then
	echo "${PROG}: ${imagedir}/${image} does not exist"
	usage
fi

for h in ${hashes}; do
	echo "${bar} ${h} hash: ${imagedir}/${image} ${bar}"
	if [ -f ${h} ]; then
		grep -v "${image}" ${h} > ${tmp}/${h} || true
	else
		cp /dev/null ${tmp}/${h}
	fi
	${cksum} -a ${h} "${image}" >> ${tmp}/${h}
	mv ${tmp}/${h} ${h}
done