[PATCH v2] aarch64: Canonicalize halving-add builtins [PR122715]
Odysseas Georgoudis <[email protected]>
| Newsgroups | gmane.comp.gcc.patches |
|---|---|
| Message-ID | <FRWP195MB28649A1B71CD5B70CF2B30E3CCD92@FRWP195MB2864.EURP195.PROD.OUTLOOK.COM> |
Thanks for pointing me to Eikansh's earlier patch and the SME issue. Would delaying the canonicalization until after inlining be an acceptable way to handle it? This keeps the target builtins visible while AArch64 IPA records their PSTATE.SM requirements, after which they can be converted to IFN_AVG_FLOOR or IFN_AVG_CEIL The attached v2 retains both the signed and unsigned canonicalizations and implements that approach. I also added a focused SME regression test for the unsigned rounding-add path. Tested with an aarch64-linux-gnu cross compiler. The targeted PR tests, the new SME test, and the existing arm_neon_1.c, arm_neon_2.c, and arm_neon_3.c tests pass. Thanks Odysseas ________________________________ From: Andrea Pinski <[email protected]> Sent: 15 August 2026 02:52 To: Odysseas Georgoudis <[email protected]> Cc: [email protected] <[email protected]> Subject: Re: [PATCH] aarch64: Canonicalize halving-add builtins [PR122715] On Fri, Aug 14, 2026 at 5:33 PM Odysseas Georgoudis <[email protected]> wrote: > > The first two patches for PR122715 have been committed. This patch > handles the remaining AArch64 case. > > Advanced SIMD halving-add intrinsics remain target builtins in GIMPLE, > preventing generic average simplifications from seeing them. Canonicalize > SHADD and UHADD to IFN_AVG_FLOOR, and SRHADD and URHADD to > IFN_AVG_CEIL. > > This allows equal operands to be folded by the existing match.pd rule > while retaining optab-based instruction selection for other operands. > > Tested with an aarch64-linux-gnu cross compiler. The targeted tests > pass. This does not fully work. In fact is is the same as Eikansh's patch (except adding the signed ones): https://inbox.sourceware.org/gcc-patches/[email protected]/ The reason why it does not work is mentioned here: https://inbox.sourceware.org/gcc-patches/CALvbMcAPZu5dupGfA=3GtS8avmzzPScYMy8SrRRQxTQhakhpKw@mail.gmail.com/ Basically gcc.target/aarch64/sme/arm_neon_1.c is no longer rejected when it should be. Eikansh was still looking into how to fix the issue mentioned but has not yet come up with a patch. He has been busy working on other things. If you want to look into how to resolve that issue that would be nice. Note the compile farm has a few aarch64 machines which you can use to do a bootstrap test. See https://gcc.gnu.org/wiki/CompileFarm on how to sign up (this is seperate from GCC but is used by many GCC developers and had been associated with GCC development for a long time now). Thanks, Andrea > > Thanks, > Odysseas >
PR122715-aarch64-v2.patch
(application/octet-stream, 7.1 KB)
From 7c32bf613deb5ea72b41a14d7a7686a378c73217 Mon Sep 17 00:00:00 2001 From: Odysseas Georgoudis <[email protected]> Date: Sat, 15 Aug 2026 23:21:35 +0100 Subject: [PATCH v2] aarch64: Canonicalize halving-add builtins [PR122715] Advanced SIMD halving-add intrinsics remain target builtins in GIMPLE, preventing generic average simplifications from seeing them. Canonicalize SHADD and UHADD to IFN_AVG_FLOOR, and SRHADD and URHADD to IFN_AVG_CEIL. Keep the builtins intact until after inlining so that AArch64 IPA can record their PSTATE.SM requirements. This preserves the required diagnostic when an Advanced SIMD intrinsic is used in a streaming context. This allows equal operands to be folded by the existing match.pd rule while retaining optab-based instruction selection for other operands. PR target/122715 gcc/ChangeLog: * config/aarch64/aarch64-builtins.cc (aarch64_general_gimple_fold_builtin): Canonicalize halving-add builtins to IFN_AVG_FLOOR and IFN_AVG_CEIL after inlining. gcc/testsuite/ChangeLog: * gcc.target/aarch64/avg-pr122715.c: New test. * gcc.target/aarch64/avg-pr122715-2.c: New test. * gcc.target/aarch64/sme/avg-pr122715.c: New test. Signed-off-by: Odysseas Georgoudis <[email protected]> --- gcc/config/aarch64/aarch64-builtins.cc | 19 +++ .../gcc.target/aarch64/avg-pr122715-2.c | 33 +++++ .../gcc.target/aarch64/avg-pr122715.c | 129 ++++++++++++++++++ .../gcc.target/aarch64/sme/avg-pr122715.c | 11 ++ 4 files changed, 192 insertions(+) create mode 100644 gcc/testsuite/gcc.target/aarch64/avg-pr122715-2.c create mode 100644 gcc/testsuite/gcc.target/aarch64/avg-pr122715.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sme/avg-pr122715.c diff --git a/gcc/config/aarch64/aarch64-builtins.cc b/gcc/config/aarch64/aarch64-builtins.cc index 8cd1bc4b1a2..60c3f6b97aa 100644 --- a/gcc/config/aarch64/aarch64-builtins.cc +++ b/gcc/config/aarch64/aarch64-builtins.cc @@ -5065,6 +5065,25 @@ aarch64_general_gimple_fold_builtin (unsigned int fcode, gcall *stmt, 1, args[0]); gimple_call_set_lhs (new_stmt, gimple_call_lhs (stmt)); break; + /* Keep Advanced SIMD halving-add builtins through IPA so that their + PSTATE.SM requirements are reflected in the function summary. Lower + them to average internal functions after inlining. */ + BUILTIN_VDQ_BHSI (BINOP, shadd, 0, DEFAULT) + BUILTIN_VDQ_BHSI (BINOPU, uhadd, 0, DEFAULT) + if (!cfun->after_inlining) + break; + new_stmt = gimple_build_call_internal (IFN_AVG_FLOOR, 2, + args[0], args[1]); + gimple_call_set_lhs (new_stmt, gimple_call_lhs (stmt)); + break; + BUILTIN_VDQ_BHSI (BINOP, srhadd, 0, DEFAULT) + BUILTIN_VDQ_BHSI (BINOPU, urhadd, 0, DEFAULT) + if (!cfun->after_inlining) + break; + new_stmt = gimple_build_call_internal (IFN_AVG_CEIL, 2, + args[0], args[1]); + gimple_call_set_lhs (new_stmt, gimple_call_lhs (stmt)); + break; BUILTIN_VSDQ_I_DI (BINOP, ashl, 3, DEFAULT) if (TREE_CODE (args[1]) == INTEGER_CST && wi::ltu_p (wi::to_wide (args[1]), element_precision (args[0]))) diff --git a/gcc/testsuite/gcc.target/aarch64/avg-pr122715-2.c b/gcc/testsuite/gcc.target/aarch64/avg-pr122715-2.c new file mode 100644 index 00000000000..c088c1327d0 --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/avg-pr122715-2.c @@ -0,0 +1,33 @@ +/* PR target/122715 */ +/* { dg-do compile } */ +/* { dg-options "-O2 -fdump-tree-optimized" } */ + +#include <arm_neon.h> + +int32x4_t avg_floor_s32(int32x4_t x, int32x4_t y) +{ + return vhaddq_s32 (x, y); +} + +uint32x4_t avg_floor_u32(uint32x4_t x, uint32x4_t y) +{ + return vhaddq_u32 (x, y); +} + +int32x4_t avg_ceil_s32(int32x4_t x, int32x4_t y) +{ + return vrhaddq_s32 (x, y); +} + +uint32x4_t avg_ceil_u32(uint32x4_t x, uint32x4_t y) +{ + return vrhaddq_u32 (x, y); +} + +/* { dg-final { scan-tree-dump-times {\.AVG_FLOOR} 2 "optimized" } } */ +/* { dg-final { scan-tree-dump-times {\.AVG_CEIL} 2 "optimized" } } */ +/* { dg-final { scan-tree-dump-not {__builtin_aarch64_} "optimized" } } */ +/* { dg-final { scan-assembler-times {\tshadd\t} 1 } } */ +/* { dg-final { scan-assembler-times {\tuhadd\t} 1 } } */ +/* { dg-final { scan-assembler-times {\tsrhadd\t} 1 } } */ +/* { dg-final { scan-assembler-times {\turhadd\t} 1 } } */ diff --git a/gcc/testsuite/gcc.target/aarch64/avg-pr122715.c b/gcc/testsuite/gcc.target/aarch64/avg-pr122715.c new file mode 100644 index 00000000000..aa46df3ffd7 --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/avg-pr122715.c @@ -0,0 +1,129 @@ +/* PR target/122715 */ +/* { dg-do compile } */ +/* { dg-options "-O2 -fdump-tree-optimized" } */ + +#include <arm_neon.h> + +int8x8_t avg_floor_s8_64(int8x8_t x) +{ + return vhadd_s8 (x, x); +} + +int16x4_t avg_floor_s16_64(int16x4_t x) +{ + return vhadd_s16 (x, x); +} + +int32x2_t avg_floor_s32_64(int32x2_t x) +{ + return vhadd_s32 (x, x); +} + +uint8x8_t avg_floor_u8_64(uint8x8_t x) +{ + return vhadd_u8 (x, x); +} + +uint16x4_t avg_floor_u16_64(uint16x4_t x) +{ + return vhadd_u16 (x, x); +} + +uint32x2_t avg_floor_u32_64(uint32x2_t x) +{ + return vhadd_u32 (x, x); +} + +int8x16_t avg_floor_s8_128(int8x16_t x) +{ + return vhaddq_s8 (x, x); +} + +int16x8_t avg_floor_s16_128(int16x8_t x) +{ + return vhaddq_s16 (x, x); +} + +int32x4_t avg_floor_s32_128(int32x4_t x) +{ + return vhaddq_s32 (x, x); +} + +uint8x16_t avg_floor_u8_128(uint8x16_t x) +{ + return vhaddq_u8 (x, x); +} + +uint16x8_t avg_floor_u16_128(uint16x8_t x) +{ + return vhaddq_u16 (x, x); +} + +uint32x4_t avg_floor_u32_128(uint32x4_t x) +{ + return vhaddq_u32 (x, x); +} + +int8x8_t avg_ceil_s8_64(int8x8_t x) +{ + return vrhadd_s8 (x, x); +} + +int16x4_t avg_ceil_s16_64(int16x4_t x) +{ + return vrhadd_s16 (x, x); +} + +int32x2_t avg_ceil_s32_64(int32x2_t x) +{ + return vrhadd_s32 (x, x); +} + +uint8x8_t avg_ceil_u8_64(uint8x8_t x) +{ + return vrhadd_u8 (x, x); +} + +uint16x4_t avg_ceil_u16_64(uint16x4_t x) +{ + return vrhadd_u16 (x, x); +} + +uint32x2_t avg_ceil_u32_64(uint32x2_t x) +{ + return vrhadd_u32 (x, x); +} + +int8x16_t avg_ceil_s8_128(int8x16_t x) +{ + return vrhaddq_s8 (x, x); +} + +int16x8_t avg_ceil_s16_128(int16x8_t x) +{ + return vrhaddq_s16 (x, x); +} + +int32x4_t avg_ceil_s32_128(int32x4_t x) +{ + return vrhaddq_s32 (x, x); +} + +uint8x16_t avg_ceil_u8_128(uint8x16_t x) +{ + return vrhaddq_u8 (x, x); +} + +uint16x8_t avg_ceil_u16_128(uint16x8_t x) +{ + return vrhaddq_u16 (x, x); +} + +uint32x4_t avg_ceil_u32_128(uint32x4_t x) +{ + return vrhaddq_u32 (x, x); +} + +/* { dg-final { scan-tree-dump-not {__builtin_aarch64_} "optimized" } } */ +/* { dg-final { scan-tree-dump-times {return x_} 24 "optimized" } } */ +/* { dg-final { scan-assembler-not {\t[su]r?hadd\t} } } */ diff --git a/gcc/testsuite/gcc.target/aarch64/sme/avg-pr122715.c b/gcc/testsuite/gcc.target/aarch64/sme/avg-pr122715.c new file mode 100644 index 00000000000..7e775686347 --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/sme/avg-pr122715.c @@ -0,0 +1,11 @@ +/* PR target/122715 */ +/* { dg-options "-O2" } */ + +#include <arm_neon.h> + +/* { dg-error {inlining failed.*'vrhaddq_u32'} "" { target *-*-* } 0 } */ +uint32x4_t avg_ceil_u32(uint32x4_t x, uint32x4_t y) + [[arm::streaming]] +{ + return vrhaddq_u32 (x, y); +} -- 2.43.5