[PATCH 09/20] alpha: add EV6/EV7 assembly memcmp

Matt Turner <[email protected]>
Newsgroups gmane.comp.lib.glibc.alpha
Message-ID <1f888a256c172062afcbe683dee419aac5d279e3.1786497497.git.mattst88@gmail.com>
Alpha had no assembly memcmp and fell back to the generic C implementation.
Add one for the alphaev6 directory (shared by the 21264 and 21364).

When both operands share alignment, byte-align to a quadword and then compare
64 bytes per iteration, OR-accumulating the per-quad xors; a non-zero
accumulator means the difference is somewhere in the block, which a byte scan
from the block base pinpoints to return the correct sign.  Operands of
differing alignment use a byte compare.

The loop issues a plain read prefetch (LDL to R31) on both streams eight cache
lines ahead.  On the 21364 (EV7), whose on-chip memory controller has a long
memory latency, this hides the Bcache misses that otherwise stall the compare;
on the 21264 it is at worst a dropped hint, so no IMPLVER dispatch is needed.
A read prefetch -- rather than the modify-intent LDS/LDT used by memset/memcpy
-- is deliberate: memcmp never writes its operands, so requesting the lines for
ownership would only add coherence traffic on a multiprocessor.

The 1 to 7 bytes left after the loops are compared with a single masked
quadword rather than a byte at a time, which would cost about eight issue
slots per byte and dominate any length that is not a multiple of eight.  Both
operands are 0mod8 by then, since the head aligned them and everything since
has advanced by whole quadwords, and an aligned quadword cannot cross a page
boundary, so a whole one can be loaded at either pointer even though only some
of it belongs to the compare; masking the difference discards the rest.
Nothing is stored, so reading past the end of the operands is not observable.
A mismatch still falls into the byte loop, which is what determines which byte
differed and in which direction.

Measured on an AlphaServer ES47 (EV7, 1.3GHz), cold cyc/call vs the generic C:
  size      generic     asm
  1024         3617    1502  2.35x
  4096        14527    6423  2.24x
  65536      231362   96424  2.40x
  262144     916913  386234  2.37x

bcmp is provided as a weak alias, as in the generic implementation.

Verified on an EV68CB against the C memcmp for every length from 0 to 300 at
24 alignments, both for equal operands and with a difference planted at every
position in turn, comparing the sign of the result.

__memcmpeq is aliased to memcmp here too.  string/memcmp.c provides that alias
for the C implementation and string/memcmpeq.c is deliberately empty, so an
assembly memcmp that replaces the C one has to define it, as sparc64 and s390
do.  Without it an --host=alphaev6 or later build links no __memcmpeq at all,
which drops a symbol the ABI lists and fails to link string/inl-tester.
---
 sysdeps/alpha/alphaev6/memcmp.S | 226 ++++++++++++++++++++++++++++++++
 1 file changed, 226 insertions(+)
 create mode 100644 sysdeps/alpha/alphaev6/memcmp.S

diff --git ./sysdeps/alpha/alphaev6/memcmp.S ./sysdeps/alpha/alphaev6/memcmp.S
new file mode 100644
index 0000000000..0a3d2e11e6
--- /dev/null
+++ ./sysdeps/alpha/alphaev6/memcmp.S
@@ -0,0 +1,226 @@
+/* Copyright (C) 2026 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library.  If not, see
+   <https://www.gnu.org/licenses/>.  */
+
+/*
+ * EV6/EV7 memcmp.  When both operands share alignment, byte-align to a
+ * quadword and compare 64 bytes per iteration, OR-accumulating the per-quad
+ * xors; a non-zero accumulator means a difference lies in the block, which a
+ * byte scan from the block base then pinpoints to return the correct sign.
+ * Operands of differing alignment fall back to a byte compare.
+ *
+ * The loop issues a plain read prefetch (LDL to R31) on both streams several
+ * cache lines ahead.  On the 21364 (EV7), whose on-chip memory controller has
+ * a long latency, this hides the Bcache misses that otherwise stall the
+ * compare; on the 21264 it is at worst a dropped hint.  A read prefetch (not
+ * the modify-intent LDS/LDT) is deliberate: memcmp never writes its operands,
+ * so requesting lines for ownership would only add coherence traffic on SMP.
+ *
+ * Temp usage:
+ *	$0	- result
+ *	$1,$2	- scratch
+ *	$3,$4	- loaded data
+ *	$5,$6	- xor / accumulator
+ */
+
+#include <sysdep.h>
+
+	.arch ev6
+	.set noreorder
+	.set noat
+
+/* Read prefetch distance, in bytes (8 cache lines ahead).  The CWG asks for
+   8/n cache blocks for n streams against the 21264's 8-entry MAF, so four for
+   the two streams here; measured on an EV68CB the curve is flat from four
+   blocks out to eight, and only a distance below two costs anything.
+
+   These must stay plain read prefetches (LDL to R31).  Asking for the lines
+   with modify intent instead (LDS to F31) measures 2x slower on an EV68CB --
+   756446 cycles against 368989 for a cold 256KB compare -- because ReadBlkMod
+   provokes about twice the Mbox replay traps on a stream that is only ever
+   read.  Bcache misses are unchanged, so it is not extra memory traffic.
+   ReadBlkModEN (LDT to F31) measures identical to the plain prefetch, since
+   the evict-next variants are 21364-only and degrade here.  */
+#define PF	(8 * 64)
+
+ENTRY(memcmp)
+	.prologue 0
+
+	mov	$31, $0			# E : default result 0 (equal)
+	beq	$18, $ret		# U : n == 0 -> equal
+	xor	$16, $17, $1		# E : do the operands share alignment?
+	and	$1, 7, $1		# E : ... mod 8
+
+	bne	$1, $bytewise		# U : different alignment -> byte compare
+	and	$16, 7, $1		# E : already 0mod8?
+	beq	$1, $aligned		# U : yes
+	nop				# E :
+
+	/* Same misalignment: byte-compare up to a 0mod8 boundary. */
+$head:
+	ldbu	$3, 0($16)		# L : a byte
+	ldbu	$4, 0($17)		# L : b byte
+	lda	$16, 1($16)		# E : a++
+	lda	$17, 1($17)		# E : b++
+
+	subq	$3, $4, $0		# E : difference (unsigned bytes)
+	bne	$0, $ret		# U : differ -> return sign
+	subq	$18, 1, $18		# E : n--
+	beq	$18, $reteq		# U : consumed all -> equal
+
+	and	$16, 7, $1		# E : aligned now?
+	bne	$1, $head		# U : keep going
+
+$aligned:
+	cmple	$18, 63, $1		# E : at least one 64B block?
+	bne	$1, $quadtail		# U : no
+	nop				# E :
+	nop				# E :
+
+	.align 4
+$loop:
+	ldl	$31, PF($16)		# L : read prefetch a, PF bytes ahead
+	ldl	$31, PF($17)		# L : read prefetch b
+	ldq	$3, 0($16)		# L : a[0..7]
+	ldq	$4, 0($17)		# L : b[0..7]
+
+	xor	$3, $4, $5		# E : seed the accumulator
+	ldq	$3, 8($16)		# L :
+	ldq	$4, 8($17)		# L :
+	xor	$3, $4, $6		# E :
+
+	or	$5, $6, $5		# E :
+	ldq	$3, 16($16)		# L :
+	ldq	$4, 16($17)		# L :
+	xor	$3, $4, $6		# E :
+
+	or	$5, $6, $5		# E :
+	ldq	$3, 24($16)		# L :
+	ldq	$4, 24($17)		# L :
+	xor	$3, $4, $6		# E :
+
+	or	$5, $6, $5		# E :
+	ldq	$3, 32($16)		# L :
+	ldq	$4, 32($17)		# L :
+	xor	$3, $4, $6		# E :
+
+	or	$5, $6, $5		# E :
+	ldq	$3, 40($16)		# L :
+	ldq	$4, 40($17)		# L :
+	xor	$3, $4, $6		# E :
+
+	or	$5, $6, $5		# E :
+	ldq	$3, 48($16)		# L :
+	ldq	$4, 48($17)		# L :
+	xor	$3, $4, $6		# E :
+
+	or	$5, $6, $5		# E :
+	ldq	$3, 56($16)		# L :
+	ldq	$4, 56($17)		# L :
+	xor	$3, $4, $6		# E :
+
+	or	$5, $6, $5		# E : full 64B xor summary
+	bne	$5, $bytewise		# U : a difference is in this block
+	subq	$18, 64, $18		# E : n -= 64
+	lda	$16, 64($16)		# E : a += 64
+
+	lda	$17, 64($17)		# E : b += 64
+	cmple	$18, 63, $1		# E : another full block?
+	beq	$1, $loop		# U :
+	nop				# E :
+
+	/* 0..63 bytes left, still 0mod8: compare remaining quads. */
+$quadtail:
+	subq	$18, 8, $2		# E : (n - 8)
+	blt	$2, $tailquad		# U : < 8 left -> masked quad
+
+$qt:
+	ldq	$3, 0($16)		# L :
+	ldq	$4, 0($17)		# L :
+	xor	$3, $4, $5		# E :
+	bne	$5, $bytewise		# U : differ in this quad (n unchanged)
+
+	lda	$16, 8($16)		# E : a += 8
+	lda	$17, 8($17)		# E : b += 8
+	mov	$2, $18			# E : commit n -= 8
+	subq	$18, 8, $2		# E : (n - 8)
+
+	bge	$2, $qt			# U : another quad
+	nop				# E :
+	nop				# E :
+	nop				# E :
+
+	/*
+	 * 1..7 bytes left, both operands 0mod8 -- the head aligned them and
+	 * everything since has advanced by whole quadwords.  An aligned
+	 * quadword cannot cross a page boundary, so loading a whole one at
+	 * either pointer is safe even though only $18 bytes belong to the
+	 * compare; masking the difference down to those bytes discards
+	 * whatever followed.  Nothing is stored, so reading past the end of
+	 * the operands is not observable.
+	 *
+	 * This is only the tail.  A mismatch still falls into the byte loop
+	 * below, which is what works out which byte differed and in which
+	 * direction, and which the block and quad loops above also branch to.
+	 */
+$tailquad:
+	beq	$18, $reteq		# U : nothing left -> equal
+	nop				# E :
+	nop				# E :
+	nop				# E :
+
+	ldq	$3, 0($16)		# L :
+	ldq	$4, 0($17)		# L :
+	xor	$3, $4, $5		# E :
+	mskql	$5, $18, $5		# U : only the bytes being compared
+
+	beq	$5, $reteq		# U : all equal
+	nop				# E :
+	nop				# E :
+	nop				# E :
+
+	/* Tail bytes, and the unaligned / locate path.  On entry from a block
+	   or quad mismatch, the first differing byte lies within the bytes
+	   scanned here, so the loop returns its sign before running off. */
+$bytewise:
+	beq	$18, $reteq		# U : nothing left -> equal
+	nop				# E :
+
+$bl:
+	ldbu	$3, 0($16)		# L :
+	ldbu	$4, 0($17)		# L :
+	lda	$16, 1($16)		# E :
+	lda	$17, 1($17)		# E :
+
+	subq	$3, $4, $0		# E : unsigned byte difference
+	bne	$0, $ret		# U : differ -> sign
+	subq	$18, 1, $18		# E : n--
+	bne	$18, $bl		# U :
+
+$reteq:
+	mov	$31, $0			# E : equal
+$ret:
+	ret	$31, ($26), 1		# L0 :
+
+END(memcmp)
+libc_hidden_builtin_def (memcmp)
+#ifdef weak_alias
+# undef bcmp
+weak_alias (memcmp, bcmp)
+#endif
+#undef __memcmpeq
+strong_alias (memcmp, __memcmpeq)
+libc_hidden_def (__memcmpeq)
-- 
2.54.0
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.