Re: [RFC] alpha: optimize ip_fast_csum for BWX-capable CPUs

> On Apr 23, 2026, at 5:30 AM, Magnus Lindholm <[email protected]> wrote:
> 
> On Mon, Apr 20, 2026 at 1:44 AM Mike Hlavac <[email protected]> wrote:
>> 
>> On Alpha EV56 and later, we can use the 'ldwu' instruction to significantly accelerate IP header checksumming. By manually unrolling the loop for the common 20-byte (ihl=5) case, we eliminate branch penalties and allow the compiler to optimally schedule instructions for the EV56 pipeline.
>> 
>> Benchmarked on EV56 (Miata) at 633MHz:
>> 
>>        • Legacy Path: 0.796s
>> 
>>        • Unrolled BWX Path: 0.508s (~36% improvement)
>> 
>> Tested with GCC 15. The unrolled C implementation results in straight-line assembly with no branches in the hot path.  I’d love some feedback from someone with an EV6+
>> 
> 
> Mike,
> 
> Thanks for working on this.  I tested the benchmark here with GCC
> 15.2.1 at -O2 on several CPU targets.  There does seem to be a real
> optimization opportunity, but I do not think this patch is the right
> shape yet.
> 
> Results here (EV67 833 MHz system) were roughly:
> 
>  ev4:  legacy ~0.365s, modern ~0.83s,  unrolled ~0.230s
>  ev5:  legacy ~0.394s, modern ~0.716s, unrolled ~0.230s
>  ev56: legacy ~0.304s, modern ~0.304s, unrolled ~0.171s
>  ev6:  legacy ~0.314s, modern ~0.313s, unrolled ~0.177s
> 
> The measured win comes from the manually unrolled ihl==5 case becoming
> straight-line code with no inner loop branch, not from the "modern C"
> rewrite by itself.  On ev56/ev6, the looped legacy and looped modern-C
> variants compile to essentially the same code, while on ev4/ev5 the
> plain uint16_t loop is actually much worse than legacy.
> 
> One caveat is that the benchmark repeatedly checksums the same small
> static header, so it mainly measures code shape with hot data.  That
> is useful, but it does not say much by itself about the end-to-end
> gain on real traffic. Touching old and stable code is alway a risk to
> introduce new problems.
> 
> I also think the interface change is a problem.  Today Alpha has an
> out-of-line ip_fast_csum() symbol.  This patch turns it into a header
> inline and makes the fallback path call do_csum(), but do_csum() is
> not exported, which looks wrong for modules.
> 
> The optimization itself does not seem out of line; other archs also
> treat ip_fast_csum() as a specialized IPv4-header routine.  What
> would seem safer here would be to keep the Alpha-specific fast path
> behind the existing ip_fast_csum() implementation, with the current
> checksum code as the fallback for larger ihl values and non-BWX
> friendly builds.
> 
> Thanks,
> Magnus

Magnus,

Thanks for the feedback.   I’ve re-worked the patch and micro-benchmark based on your comments.   The benchmark has more overhead from the XOR operations, but it gives a different view of the same picture with a non-static header.

Benchmark results on my machine (updated version):

Legacy Path: 2.648437 seconds
Unrolled C Path: 2.429688 seconds

Updated benchmark:

#include <stdio.h>
#include <time.h>
#include <stdint.h>

//generate more randomized header values
static inline uint32_t xorshift32(uint32_t *state)
{
    uint32_t x = *state;
    x ^= x << 13;
    x ^= x >> 17;
    x ^= x << 5;
    return *state = x;
}

// Current Kernel-style Logic (Simplified)
// This simulates the ldq_u / extwl dance needed by ev4
uint16_t csum_legacy(const uint16_t *iph) {
   uint64_t sum = 0;
   // Simulate the overhead of checking alignment and doing shifts
   // This is essentially what do_csum does for a 20-byte header
   for (int i = 0; i < 10; i++) {
       sum += iph[i];
   }
   while (sum >> 16)
       sum = (sum & 0xffff) + (sum >> 16);
   return (uint16_t)~sum;
}

uint16_t csum_unrolled_c(const void *iph) {
   const uint16_t *w = (const uint16_t *)iph;
   uint64_t sum;

   sum =  w[0];
   sum += w[1];
   sum += w[2];
   sum += w[3];
   sum += w[4];
   sum += w[5];
   sum += w[6];
   sum += w[7];
   sum += w[8];
   sum += w[9];

   uint64_t tmp = (sum & 0xffff) + (sum >> 16);
   tmp = (tmp & 0xffff) + (tmp >> 16);

   return (uint16_t)~tmp;
}

int main() {
   uint16_t header[10] = {0x4500, 0x003c, 0x1c46, 0x4000, 0x4006, 0x0000, 0xac10, 0x0a63, 0xac10, 0x0a0c};
   uint16_t header2[10] = {0x4500, 0x003c, 0x1c46, 0x4000, 0x4006, 0x0000, 0xac10, 0x0a63, 0xac10, 0x0a0c};
   long iterations = 10000000;
   uint32_t rng_state = 0x12345678;
   clock_t start, end;

// Test Legacy
   start = clock();
   for (long i = 0; i < iterations; i++) {
       for (int j = 0; j < 10; j++)
           header[j] ^= (uint16_t)xorshift32(&rng_state);
       volatile uint16_t res = csum_legacy(header);
   }
   end = clock();
   printf("Legacy Path: %f seconds\n", (double)(end - start) / CLOCKS_PER_SEC);

   rng_state = 0x12345678;

   // Test Unrolled
   start = clock();
   for (long i = 0; i < iterations; i++) {
       for (int j = 0; j < 10; j++)
           header2[j] ^= (uint16_t)xorshift32(&rng_state);
       volatile uint16_t res = csum_unrolled_c(header2);
   }
   end = clock();

   printf("Unrolled C Path: %f seconds\n", (double)(end - start) / CLOCKS_PER_SEC);
   return 0;
}

Updated patch:

Signed-off-by: Mike Hlavac <[email protected]>
Assisted-by: Google Gemini

--- /home/griffin/kernel-hacking/checksum/checksum-orig.c	2026-04-12 15:17:48.062299877 -0400
+++ arch/alpha/lib/checksum.c	2026-05-13 02:41:15.379142990 -0400
@@ -16,6 +16,7 @@

 #include <asm/byteorder.h>
 #include <asm/checksum.h>
+#include <asm/special_insns.h>

 static inline unsigned short from64to16(unsigned long x)
 {
@@ -142,12 +143,38 @@

 /*
  *	This is a version of ip_compute_csum() optimized for IP headers,
- *	which always checksum on 4 octet boundaries.
+ *	which always checksum on 4 octet boundaries.   Optimized version
+ *	provided for CPUs that see benefit for the unrolled logic.
  */
 __sum16 ip_fast_csum(const void *iph, unsigned int ihl)
 {
+    if (likely(ihl == 5) && !amask(AMASK_BWX)) {
+        const u16 *w = iph;
+        u64 sum;
+
+        sum  = w[0];
+        sum += w[1];
+        sum += w[2];
+        sum += w[3];
+        sum += w[4];
+        sum += w[5];
+        sum += w[6];
+        sum += w[7];
+        sum += w[8];
+        sum += w[9];
+
+        sum = (sum & 0xffff) + (sum >> 16);
+        sum = (sum & 0xffff) + (sum >> 16);
+
+        return (__force __sum16)~sum;
+    }
+	/* existing legacy implementation as fallback for:
+	*   - ihl != 5
+	*   - EV4/EV5 where the old code is better
+	*/
 	return (__force __sum16)~do_csum(iph,ihl*4);
 }
+
 EXPORT_SYMBOL(ip_fast_csum);

 /*