[PATCH] aarch64: Add SME2p3/SVE2p3 x2 narrowing right shift intrinsics
Abhishek Kaushik <[email protected]>
| Newsgroups | gmane.comp.gcc.patches |
|---|---|
| Message-ID | <[email protected]> |
This patch adds support for the following SME2p3/SVE2p3 x2 narrowing
right shift intrinsics:
- svqrshrn_s8_s16
- svqrshrn_u8_u16
- svqrshrun_u8_s16
- svqshrn_s8_s16
- svqshrn_s16_s32
- svqshrn_u8_u16
- svqshrn_u16_u32
- svqshrun_u8_s16
- svqshrun_u16_s32
Bootstrapped and regression tested on aarch64-linux-gnu.
gcc/ChangeLog:
* config/aarch64/aarch64-acle-builtins.h
(TYPES_qshr_x2_sve2p3): New macro.
(TYPES_qrshr_x2_sve2p3): Likewise.
(TYPES_qshru_x2_sve2p3): Likewise.
(TYPES_qrshrun_x2_sve2p3): Likewise.
(DEF_SVE_TYPES_ARRAY): Add qshr_x2_sve2p3,
qrshr_x2_sve2p3, qrshrun_x2_sve2p3 and qshru_x2_sve2p3.
* config/aarch64/aarch64-sve-builtins-sve2.cc (svqshrn): New
function.
(svqshrun): Likewise.
* config/aarch64/aarch64-sve-builtins-sve2.def (svqrshrn): Add
SVE2p3/SME2p3 x2 form.
(svqrshrun): Likewise.
(svqshrn): Add SVE2p3/SME2p3 x2 form.
(svqshrun): Likewise.
* config/aarch64/aarch64-sve-builtins-sve2.h (svqshrn): Declare.
(svqshrun): Likewise.
* config/aarch64/aarch64-sve2.md
(@aarch64_sve_<sve_int_op><mode>): New define_insn.
* config/aarch64/iterators.md (SVE_FULL_HIx2): New mode iterator.
(UNSPEC_SQSHRN, UNSPEC_SQSHRUN, UNSPEC_UQSHRN): New unspecs.
(VNARROW): Add VNx16HI mapping.
(Ventype): Likewise.
(SVE2_INT_SHIFT_IMM_NARROWxN): Add UNSPEC_SQSHRN,
UNSPEC_SQSHRUN and UNSPEC_UQSHRN.
(sve_int_op): Add sqshrn, sqshrun and uqshrn.
gcc/testsuite/ChangeLog:
* gcc.target/aarch64/sme2/acle-asm/qrshrn_s8_x2.c: New test.
* gcc.target/aarch64/sme2/acle-asm/qrshrn_u8_x2.c: New test.
* gcc.target/aarch64/sme2/acle-asm/qrshrun_u8_x2.c: New test.
* gcc.target/aarch64/sme2/acle-asm/qshrn_s16_x2.c: New test.
* gcc.target/aarch64/sme2/acle-asm/qshrn_s8_x2.c: New test.
* gcc.target/aarch64/sme2/acle-asm/qshrn_u16_x2.c: New test.
* gcc.target/aarch64/sme2/acle-asm/qshrn_u8_x2.c: New test.
* gcc.target/aarch64/sme2/acle-asm/qshrun_u16_x2.c: New test.
* gcc.target/aarch64/sme2/acle-asm/qshrun_u8_x2.c: New test.
---
gcc/config/aarch64/aarch64-acle-builtins.h | 30 +++++++++++
.../aarch64/aarch64-sve-builtins-sve2.cc | 6 +++
.../aarch64/aarch64-sve-builtins-sve2.def | 24 ++++++---
.../aarch64/aarch64-sve-builtins-sve2.h | 2 +
gcc/config/aarch64/aarch64-sve2.md | 14 +++++
gcc/config/aarch64/iterators.md | 15 +++++-
.../aarch64/sme2/acle-asm/qrshrn_s8_x2.c | 53 +++++++++++++++++++
.../aarch64/sme2/acle-asm/qrshrn_u8_x2.c | 53 +++++++++++++++++++
.../aarch64/sme2/acle-asm/qrshrun_u8_x2.c | 53 +++++++++++++++++++
.../aarch64/sme2/acle-asm/qshrn_s16_x2.c | 53 +++++++++++++++++++
.../aarch64/sme2/acle-asm/qshrn_s8_x2.c | 53 +++++++++++++++++++
.../aarch64/sme2/acle-asm/qshrn_u16_x2.c | 53 +++++++++++++++++++
.../aarch64/sme2/acle-asm/qshrn_u8_x2.c | 53 +++++++++++++++++++
.../aarch64/sme2/acle-asm/qshrun_u16_x2.c | 53 +++++++++++++++++++
.../aarch64/sme2/acle-asm/qshrun_u8_x2.c | 53 +++++++++++++++++++
15 files changed, 558 insertions(+), 10 deletions(-)
create mode 100644 gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qrshrn_s8_x2.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qrshrn_u8_x2.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qrshrun_u8_x2.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qshrn_s16_x2.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qshrn_s8_x2.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qshrn_u16_x2.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qshrn_u8_x2.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qshrun_u16_x2.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qshrun_u8_x2.c
diff --git a/gcc/config/aarch64/aarch64-acle-builtins.h b/gcc/config/aarch64/aarch64-acle-builtins.h
index 42150a2632b..c332fec60f5 100644
--- a/gcc/config/aarch64/aarch64-acle-builtins.h
+++ b/gcc/config/aarch64/aarch64-acle-builtins.h
@@ -1580,10 +1580,36 @@ function_expander::result_mode () const
D (s16, s32), \
D (u16, u32)
+/* _s8_s16
+ _u8_u16
+ _s16_s32
+ _u16_u32. */
+#define TYPES_qshr_x2_sve2p3(S, D, T) \
+ D (s8, s16), \
+ D (u8, u16), \
+ D (s16, s32), \
+ D (u16, u32)
+
+/* _s8_s16
+ _u8_u16. */
+#define TYPES_qrshr_x2_sve2p3(S, D, T) \
+ D (s8, s16), \
+ D (u8, u16)
+
/* _u16_s32. */
#define TYPES_qrshru_x2(S, D, T) \
D (u16, s32)
+/* _u8_s16
+ _u16_s32. */
+#define TYPES_qshru_x2_sve2p3(S, D, T) \
+ D (u8, s16), \
+ D (u16, s32)
+
+/* _u8_s16. */
+#define TYPES_qrshrun_x2_sve2p3(S, D, T) \
+ D (u8, s16)
+
/* _s8_s32
_s16_s64
_u8_u32
@@ -2039,9 +2065,13 @@ DEF_SVE_TYPES_ARRAY (cvttb);
DEF_SVE_TYPES_ARRAY (inc_dec_n);
DEF_SVE_TYPES_ARRAY (qcvt_x2);
DEF_SVE_TYPES_ARRAY (qcvt_x4);
+DEF_SVE_TYPES_ARRAY (qshr_x2_sve2p3);
DEF_SVE_TYPES_ARRAY (qrshr_x2);
+DEF_SVE_TYPES_ARRAY (qrshr_x2_sve2p3);
DEF_SVE_TYPES_ARRAY (qrshr_x4);
DEF_SVE_TYPES_ARRAY (qrshru_x2);
+DEF_SVE_TYPES_ARRAY (qrshrun_x2_sve2p3);
+DEF_SVE_TYPES_ARRAY (qshru_x2_sve2p3);
DEF_SVE_TYPES_ARRAY (qrshru_x4);
DEF_SVE_TYPES_ARRAY (reinterpret);
DEF_SVE_TYPES_ARRAY (reinterpret_b);
diff --git a/gcc/config/aarch64/aarch64-sve-builtins-sve2.cc b/gcc/config/aarch64/aarch64-sve-builtins-sve2.cc
index c22b5b176be..520a96cecd0 100644
--- a/gcc/config/aarch64/aarch64-sve-builtins-sve2.cc
+++ b/gcc/config/aarch64/aarch64-sve-builtins-sve2.cc
@@ -1250,6 +1250,12 @@ FUNCTION (svqrshr, unspec_based_uncond_function, (UNSPEC_SQRSHR,
UNSPEC_UQRSHR, UNSPEC_NONE, UNSPEC_NONE, 1))
FUNCTION (svqrshrn, unspec_based_uncond_function, (UNSPEC_SQRSHRN,
UNSPEC_UQRSHRN, UNSPEC_NONE, UNSPEC_NONE, 1))
+FUNCTION (svqshrn, unspec_based_uncond_function, (UNSPEC_SQSHRN,
+ UNSPEC_UQSHRN, UNSPEC_NONE,
+ UNSPEC_NONE, 1))
+FUNCTION (svqshrun, unspec_based_uncond_function, (UNSPEC_SQSHRUN,
+ UNSPEC_NONE, UNSPEC_NONE,
+ UNSPEC_NONE, 1))
FUNCTION (svqrshrnb, unspec_based_function, (UNSPEC_SQRSHRNB, UNSPEC_UQRSHRNB))
FUNCTION (svqrshrnt, unspec_based_function, (UNSPEC_SQRSHRNT, UNSPEC_UQRSHRNT))
FUNCTION (svqrshru, unspec_based_uncond_function, (UNSPEC_SQRSHRU, UNSPEC_NONE, UNSPEC_NONE, UNSPEC_NONE, 1))
diff --git a/gcc/config/aarch64/aarch64-sve-builtins-sve2.def b/gcc/config/aarch64/aarch64-sve-builtins-sve2.def
index 87f7b472dd9..cc8c1043b67 100644
--- a/gcc/config/aarch64/aarch64-sve-builtins-sve2.def
+++ b/gcc/config/aarch64/aarch64-sve-builtins-sve2.def
@@ -326,6 +326,22 @@ DEF_SVE_FUNCTION (svrint64x, unary, sd_float, mxz)
DEF_SVE_FUNCTION (svrint64z, unary, sd_float, mxz)
#undef REQUIRED_EXTENSIONS
+#define REQUIRED_EXTENSIONS sve_and_sme (AARCH64_FL_SVE2p3, AARCH64_FL_SME2p3)
+DEF_SVE_FUNCTION (svdot, ternary_qq_opt_n_or_011, s_narrow_su, none)
+DEF_SVE_FUNCTION (svdot_lane, ternary_qq_or_011_lane, s_narrow_su, none)
+DEF_SVE_FUNCTION_GS (svcvtn, unary_convertxn, cvtzn, x2, none)
+DEF_SVE_FUNCTION (svcvtt, unary_convert, cvttb, none)
+DEF_SVE_FUNCTION (svcvtb, unary_convert, cvttb, none)
+DEF_SVE_FUNCTION_GS (svqrshrn, shift_right_imm_narrowxn,
+ qrshr_x2_sve2p3, x2, none)
+DEF_SVE_FUNCTION_GS (svqrshrun, shift_right_imm_narrowxn,
+ qrshrun_x2_sve2p3, x2, none)
+DEF_SVE_FUNCTION_GS (svqshrn, shift_right_imm_narrowxn,
+ qshr_x2_sve2p3, x2, none)
+DEF_SVE_FUNCTION_GS (svqshrun, shift_right_imm_narrowxn,
+ qshru_x2_sve2p3, x2, none)
+#undef REQUIRED_EXTENSIONS
+
#define REQUIRED_EXTENSIONS streaming_only (AARCH64_FL_SME2)
DEF_SVE_FUNCTION_GS (svadd, binary_single, all_integer, x24, none)
DEF_SVE_FUNCTION_GS (svclamp, clamp, all_arith, x24, none)
@@ -475,14 +491,6 @@ DEF_SVE_FUNCTION_GS_FPM (svmmla, mmla, h_float_mf8, none, none, set)
DEF_SVE_FUNCTION_GS_FPM (svmmla, mmla, s_float_mf8, none, none, set)
#undef REQUIRED_EXTENSIONS
-#define REQUIRED_EXTENSIONS sve_and_sme (AARCH64_FL_SVE2p3, AARCH64_FL_SME2p3)
-DEF_SVE_FUNCTION (svdot, ternary_qq_opt_n_or_011, s_narrow_su, none)
-DEF_SVE_FUNCTION (svdot_lane, ternary_qq_or_011_lane, s_narrow_su, none)
-DEF_SVE_FUNCTION_GS (svcvtn, unary_convertxn, cvtzn, x2, none)
-DEF_SVE_FUNCTION (svcvtt, unary_convert, cvttb, none)
-DEF_SVE_FUNCTION (svcvtb, unary_convert, cvttb, none)
-#undef REQUIRED_EXTENSIONS
-
#define REQUIRED_EXTENSIONS nonstreaming_sve (AARCH64_FL_SVE_F16F32MM)
DEF_SVE_FUNCTION (svmmla, mmla, cvt_f32_f16, none)
#undef REQUIRED_EXTENSIONS
diff --git a/gcc/config/aarch64/aarch64-sve-builtins-sve2.h b/gcc/config/aarch64/aarch64-sve-builtins-sve2.h
index 0cde240403c..2b363b4f4d5 100644
--- a/gcc/config/aarch64/aarch64-sve-builtins-sve2.h
+++ b/gcc/config/aarch64/aarch64-sve-builtins-sve2.h
@@ -184,6 +184,8 @@ namespace aarch64_acle
extern const function_base *const svqrdmulh_lane;
extern const function_base *const svqrshl;
extern const function_base *const svqrshr;
+ extern const function_base *const svqshrn;
+ extern const function_base *const svqshrun;
extern const function_base *const svqrshrn;
extern const function_base *const svqrshrnb;
extern const function_base *const svqrshrnt;
diff --git a/gcc/config/aarch64/aarch64-sve2.md b/gcc/config/aarch64/aarch64-sve2.md
index fe6aa65823d..2189e88374f 100644
--- a/gcc/config/aarch64/aarch64-sve2.md
+++ b/gcc/config/aarch64/aarch64-sve2.md
@@ -3219,10 +3219,13 @@
;; ---- [INT] Multi-vector narrowing right shifts
;; -------------------------------------------------------------------------
;; Includes:
+;; - SQSHRN (SVE2p3, SME2p3)
+;; - SQSHRUN (SVE2p3, SME2p3)
;; - SQRSHR (SME2)
;; - SQRSHRN (SVE2p1, SME2)
;; - SQRSHRU (SME2)
;; - SQRSHRUN (SVE2p1, SME2)
+;; - UQSHRN (SVE2p3, SME2p3)
;; - UQRSHR (SME2)
;; - UQRSHRN (SVE2p1, SME2)
;; -------------------------------------------------------------------------
@@ -3238,6 +3241,17 @@
[(set_attr "sve_type" "sve_int_shift")]
)
+(define_insn "@aarch64_sve_<sve_int_op><mode>"
+ [(set (match_operand:<VNARROW> 0 "register_operand" "=w")
+ (unspec:<VNARROW>
+ [(match_operand:SVE_FULL_HIx2 1 "register_operand" "Uw<vector_count>")
+ (match_operand:DI 2 "const_int_operand")]
+ SVE2_INT_SHIFT_IMM_NARROWxN))]
+ "TARGET_SVE2p3_OR_SME2p3"
+ "<sve_int_op>\t%0.<Ventype>, %1, #%2"
+ [(set_attr "sve_type" "sve_int_shift")]
+)
+
;; =========================================================================
;; == Pairwise arithmetic
;; =========================================================================
diff --git a/gcc/config/aarch64/iterators.md b/gcc/config/aarch64/iterators.md
index e7ae93d1896..dc7bb22b89b 100644
--- a/gcc/config/aarch64/iterators.md
+++ b/gcc/config/aarch64/iterators.md
@@ -626,6 +626,8 @@
;; 2x and 4x tuples of the above, excluding 2x DI.
(define_mode_iterator SVE_FULL_SIx2_SDIx4 [VNx8SI VNx16SI VNx8DI])
+(define_mode_iterator SVE_FULL_HIx2 [VNx16HI])
+
;; Fully-packed SVE floating-point vector modes that have 32-bit or 64-bit
;; elements.
(define_mode_iterator SVE_FULL_SDF [VNx4SF VNx2DF])
@@ -1214,6 +1216,8 @@
UNSPEC_SQRDCMLAH270 ; Used in aarch64-sve2.md.
UNSPEC_SQRDCMLAH90 ; Used in aarch64-sve2.md.
UNSPEC_SQRSHR ; Used in aarch64-sve2.md.
+ UNSPEC_SQSHRN ; Used in aarch64-sve2.md.
+ UNSPEC_SQSHRUN ; Used in aarch64-sve2.md.
UNSPEC_SQRSHRN ; Used in aarch64-sve2.md.
UNSPEC_SQRSHRNB ; Used in aarch64-sve2.md.
UNSPEC_SQRSHRNT ; Used in aarch64-sve2.md.
@@ -1257,6 +1261,7 @@
UNSPEC_UMULLB ; Used in aarch64-sve2.md.
UNSPEC_UMULLT ; Used in aarch64-sve2.md.
UNSPEC_UQRSHR ; Used in aarch64-sve2.md.
+ UNSPEC_UQSHRN ; Used in aarch64-sve2.md.
UNSPEC_UQRSHRN ; Used in aarch64-sve2.md.
UNSPEC_UQRSHRNB ; Used in aarch64-sve2.md.
UNSPEC_UQRSHRNT ; Used in aarch64-sve2.md.
@@ -2152,7 +2157,7 @@
(VNx4SI "VNx8HI") (VNx4SF "VNx8HF")
(VNx2DI "VNx4SI") (VNx2DF "VNx4SF")
(VNx8SI "VNx8HI") (VNx16SI "VNx16QI")
- (VNx8DI "VNx8HI")])
+ (VNx8DI "VNx8HI") (VNx16HI "VNx16QI")])
(define_mode_attr Vnarrow [(VNx8HI "vnx16qi")
(VNx4SI "vnx8hi") (VNx4SF "vnx8hf")
(VNx2DI "vnx4si") (VNx2DF "vnx4sf")
@@ -2293,7 +2298,7 @@
(VNx4SI "h") (VNx4SF "h")
(VNx2DI "s") (VNx2DF "s")
(VNx8SI "h") (VNx16SI "b")
- (VNx8DI "h")])
+ (VNx8DI "h") (VNx16HI "b")])
;; SVE vector after widening.
(define_mode_attr Vewtype [(VNx16QI "h")
@@ -4073,10 +4078,13 @@
(define_int_iterator SVE2_INT_SHIFT_IMM_NARROWxN
[(UNSPEC_SQRSHR "TARGET_STREAMING_SME2")
+ (UNSPEC_SQSHRN "TARGET_SVE2p3_OR_SME2p3")
+ (UNSPEC_SQSHRUN "TARGET_SVE2p3_OR_SME2p3")
(UNSPEC_SQRSHRN "TARGET_SVE2p1_OR_SME2")
(UNSPEC_SQRSHRU "TARGET_STREAMING_SME2")
(UNSPEC_SQRSHRUN "TARGET_SVE2p1_OR_SME2")
(UNSPEC_UQRSHR "TARGET_STREAMING_SME2")
+ (UNSPEC_UQSHRN "TARGET_SVE2p3_OR_SME2p3")
(UNSPEC_UQRSHRN "TARGET_SVE2p1_OR_SME2")])
(define_int_iterator SVE2_INT_SHIFT_INSERT [UNSPEC_SLI UNSPEC_SRI])
@@ -4967,6 +4975,8 @@
(UNSPEC_SQRDMULH "sqrdmulh")
(UNSPEC_SQRSHL "sqrshl")
(UNSPEC_SQRSHR "sqrshr")
+ (UNSPEC_SQSHRN "sqshrn")
+ (UNSPEC_SQSHRUN "sqshrun")
(UNSPEC_SQRSHRN "sqrshrn")
(UNSPEC_SQRSHRNB "sqrshrnb")
(UNSPEC_SQRSHRNT "sqrshrnt")
@@ -5017,6 +5027,7 @@
(UNSPEC_UMULLT "umullt")
(UNSPEC_UQRSHL "uqrshl")
(UNSPEC_UQRSHR "uqrshr")
+ (UNSPEC_UQSHRN "uqshrn")
(UNSPEC_UQRSHRN "uqrshrn")
(UNSPEC_UQRSHRNB "uqrshrnb")
(UNSPEC_UQRSHRNT "uqrshrnt")
diff --git a/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qrshrn_s8_x2.c b/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qrshrn_s8_x2.c
new file mode 100644
index 00000000000..049db7d8690
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qrshrn_s8_x2.c
@@ -0,0 +1,53 @@
+/* { dg-do assemble { target aarch64_asm_sme2p3_ok } } */
+/* { dg-do compile { target { ! aarch64_asm_sme2p3_ok } } } */
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sme2_acle.h"
+#pragma GCC target "+sme2p3"
+
+/*
+** qrshrn_z0_z0_1:
+** sqrshrn z0\.b, {z0\.h - z1\.h}, #1
+** ret
+*/
+TEST_X2_NARROW (qrshrn_z0_z0_1, svint16x2_t, svint8_t,
+ z0_res = svqrshrn_n_s8_s16_x2 (z0, 1),
+ z0_res = svqrshrn_s8 (z0, 1))
+
+/*
+** qrshrn_z0_z6_8:
+** sqrshrn z0\.b, {z6\.h - z7\.h}, #8
+** ret
+*/
+TEST_X2_NARROW (qrshrn_z0_z6_8, svint16x2_t, svint8_t,
+ z0_res = svqrshrn_n_s8_s16_x2 (z6, 8),
+ z0_res = svqrshrn_s8 (z6, 8))
+
+/*
+** qrshrn_z0_z29_5:
+** mov [^\n]+
+** mov [^\n]+
+** sqrshrn z0\.b, [^\n]+, #5
+** ret
+*/
+TEST_X2_NARROW (qrshrn_z0_z29_5, svint16x2_t, svint8_t,
+ z0_res = svqrshrn_n_s8_s16_x2 (z29, 5),
+ z0_res = svqrshrn_s8 (z29, 5))
+
+/*
+** qrshrn_z5_z0_3:
+** sqrshrn z5\.b, {z0\.h - z1\.h}, #3
+** ret
+*/
+TEST_X2_NARROW (qrshrn_z5_z0_3, svint16x2_t, svint8_t,
+ z5 = svqrshrn_n_s8_s16_x2 (z0, 3),
+ z5 = svqrshrn_s8 (z0, 3))
+
+/*
+** qrshrn_z22_z16_7:
+** sqrshrn z22\.b, {z16\.h - z17\.h}, #7
+** ret
+*/
+TEST_X2_NARROW (qrshrn_z22_z16_7, svint16x2_t, svint8_t,
+ z22 = svqrshrn_n_s8_s16_x2 (z16, 7),
+ z22 = svqrshrn_s8 (z16, 7))
diff --git a/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qrshrn_u8_x2.c b/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qrshrn_u8_x2.c
new file mode 100644
index 00000000000..974deec9fe3
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qrshrn_u8_x2.c
@@ -0,0 +1,53 @@
+/* { dg-do assemble { target aarch64_asm_sme2p3_ok } } */
+/* { dg-do compile { target { ! aarch64_asm_sme2p3_ok } } } */
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sme2_acle.h"
+#pragma GCC target "+sme2p3"
+
+/*
+** qrshrn_z0_z0_1:
+** uqrshrn z0\.b, {z0\.h - z1\.h}, #1
+** ret
+*/
+TEST_X2_NARROW (qrshrn_z0_z0_1, svuint16x2_t, svuint8_t,
+ z0_res = svqrshrn_n_u8_u16_x2 (z0, 1),
+ z0_res = svqrshrn_u8 (z0, 1))
+
+/*
+** qrshrn_z0_z6_8:
+** uqrshrn z0\.b, {z6\.h - z7\.h}, #8
+** ret
+*/
+TEST_X2_NARROW (qrshrn_z0_z6_8, svuint16x2_t, svuint8_t,
+ z0_res = svqrshrn_n_u8_u16_x2 (z6, 8),
+ z0_res = svqrshrn_u8 (z6, 8))
+
+/*
+** qrshrn_z0_z29_5:
+** mov [^\n]+
+** mov [^\n]+
+** uqrshrn z0\.b, [^\n]+, #5
+** ret
+*/
+TEST_X2_NARROW (qrshrn_z0_z29_5, svuint16x2_t, svuint8_t,
+ z0_res = svqrshrn_n_u8_u16_x2 (z29, 5),
+ z0_res = svqrshrn_u8 (z29, 5))
+
+/*
+** qrshrn_z5_z0_3:
+** uqrshrn z5\.b, {z0\.h - z1\.h}, #3
+** ret
+*/
+TEST_X2_NARROW (qrshrn_z5_z0_3, svuint16x2_t, svuint8_t,
+ z5 = svqrshrn_n_u8_u16_x2 (z0, 3),
+ z5 = svqrshrn_u8 (z0, 3))
+
+/*
+** qrshrn_z22_z16_7:
+** uqrshrn z22\.b, {z16\.h - z17\.h}, #7
+** ret
+*/
+TEST_X2_NARROW (qrshrn_z22_z16_7, svuint16x2_t, svuint8_t,
+ z22 = svqrshrn_n_u8_u16_x2 (z16, 7),
+ z22 = svqrshrn_u8 (z16, 7))
diff --git a/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qrshrun_u8_x2.c b/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qrshrun_u8_x2.c
new file mode 100644
index 00000000000..8834c38a160
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qrshrun_u8_x2.c
@@ -0,0 +1,53 @@
+/* { dg-do assemble { target aarch64_asm_sme2p3_ok } } */
+/* { dg-do compile { target { ! aarch64_asm_sme2p3_ok } } } */
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sme2_acle.h"
+#pragma GCC target "+sme2p3"
+
+/*
+** qrshrun_z0_z0_1:
+** sqrshrun z0\.b, {z0\.h - z1\.h}, #1
+** ret
+*/
+TEST_X2_NARROW (qrshrun_z0_z0_1, svint16x2_t, svuint8_t,
+ z0_res = svqrshrun_n_u8_s16_x2 (z0, 1),
+ z0_res = svqrshrun_u8 (z0, 1))
+
+/*
+** qrshrun_z0_z6_8:
+** sqrshrun z0\.b, {z6\.h - z7\.h}, #8
+** ret
+*/
+TEST_X2_NARROW (qrshrun_z0_z6_8, svint16x2_t, svuint8_t,
+ z0_res = svqrshrun_n_u8_s16_x2 (z6, 8),
+ z0_res = svqrshrun_u8 (z6, 8))
+
+/*
+** qrshrun_z0_z29_5:
+** mov [^\n]+
+** mov [^\n]+
+** sqrshrun z0\.b, [^\n]+, #5
+** ret
+*/
+TEST_X2_NARROW (qrshrun_z0_z29_5, svint16x2_t, svuint8_t,
+ z0_res = svqrshrun_n_u8_s16_x2 (z29, 5),
+ z0_res = svqrshrun_u8 (z29, 5))
+
+/*
+** qrshrun_z5_z0_3:
+** sqrshrun z5\.b, {z0\.h - z1\.h}, #3
+** ret
+*/
+TEST_X2_NARROW (qrshrun_z5_z0_3, svint16x2_t, svuint8_t,
+ z5 = svqrshrun_n_u8_s16_x2 (z0, 3),
+ z5 = svqrshrun_u8 (z0, 3))
+
+/*
+** qrshrun_z22_z16_7:
+** sqrshrun z22\.b, {z16\.h - z17\.h}, #7
+** ret
+*/
+TEST_X2_NARROW (qrshrun_z22_z16_7, svint16x2_t, svuint8_t,
+ z22 = svqrshrun_n_u8_s16_x2 (z16, 7),
+ z22 = svqrshrun_u8 (z16, 7))
diff --git a/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qshrn_s16_x2.c b/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qshrn_s16_x2.c
new file mode 100644
index 00000000000..fcfc7d3c55c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qshrn_s16_x2.c
@@ -0,0 +1,53 @@
+/* { dg-do assemble { target aarch64_asm_sme2p3_ok } } */
+/* { dg-do compile { target { ! aarch64_asm_sme2p3_ok } } } */
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sme2_acle.h"
+#pragma GCC target "+sme2p3"
+
+/*
+** qshrn_z0_z0_1:
+** sqshrn z0\.h, {z0\.s - z1\.s}, #1
+** ret
+*/
+TEST_X2_NARROW (qshrn_z0_z0_1, svint32x2_t, svint16_t,
+ z0_res = svqshrn_n_s16_s32_x2 (z0, 1),
+ z0_res = svqshrn_s16 (z0, 1))
+
+/*
+** qshrn_z0_z6_16:
+** sqshrn z0\.h, {z6\.s - z7\.s}, #16
+** ret
+*/
+TEST_X2_NARROW (qshrn_z0_z6_16, svint32x2_t, svint16_t,
+ z0_res = svqshrn_n_s16_s32_x2 (z6, 16),
+ z0_res = svqshrn_s16 (z6, 16))
+
+/*
+** qshrn_z0_z29_13:
+** mov [^\n]+
+** mov [^\n]+
+** sqshrn z0\.h, [^\n]+, #13
+** ret
+*/
+TEST_X2_NARROW (qshrn_z0_z29_13, svint32x2_t, svint16_t,
+ z0_res = svqshrn_n_s16_s32_x2 (z29, 13),
+ z0_res = svqshrn_s16 (z29, 13))
+
+/*
+** qshrn_z5_z0_11:
+** sqshrn z5\.h, {z0\.s - z1\.s}, #11
+** ret
+*/
+TEST_X2_NARROW (qshrn_z5_z0_11, svint32x2_t, svint16_t,
+ z5 = svqshrn_n_s16_s32_x2 (z0, 11),
+ z5 = svqshrn_s16 (z0, 11))
+
+/*
+** qshrn_z22_z16_15:
+** sqshrn z22\.h, {z16\.s - z17\.s}, #15
+** ret
+*/
+TEST_X2_NARROW (qshrn_z22_z16_15, svint32x2_t, svint16_t,
+ z22 = svqshrn_n_s16_s32_x2 (z16, 15),
+ z22 = svqshrn_s16 (z16, 15))
diff --git a/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qshrn_s8_x2.c b/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qshrn_s8_x2.c
new file mode 100644
index 00000000000..c006bbcff56
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qshrn_s8_x2.c
@@ -0,0 +1,53 @@
+/* { dg-do assemble { target aarch64_asm_sme2p3_ok } } */
+/* { dg-do compile { target { ! aarch64_asm_sme2p3_ok } } } */
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sme2_acle.h"
+#pragma GCC target "+sme2p3"
+
+/*
+** qshrn_z0_z0_1:
+** sqshrn z0\.b, {z0\.h - z1\.h}, #1
+** ret
+*/
+TEST_X2_NARROW (qshrn_z0_z0_1, svint16x2_t, svint8_t,
+ z0_res = svqshrn_n_s8_s16_x2 (z0, 1),
+ z0_res = svqshrn_s8 (z0, 1))
+
+/*
+** qshrn_z0_z6_8:
+** sqshrn z0\.b, {z6\.h - z7\.h}, #8
+** ret
+*/
+TEST_X2_NARROW (qshrn_z0_z6_8, svint16x2_t, svint8_t,
+ z0_res = svqshrn_n_s8_s16_x2 (z6, 8),
+ z0_res = svqshrn_s8 (z6, 8))
+
+/*
+** qshrn_z0_z29_5:
+** mov [^\n]+
+** mov [^\n]+
+** sqshrn z0\.b, [^\n]+, #5
+** ret
+*/
+TEST_X2_NARROW (qshrn_z0_z29_5, svint16x2_t, svint8_t,
+ z0_res = svqshrn_n_s8_s16_x2 (z29, 5),
+ z0_res = svqshrn_s8 (z29, 5))
+
+/*
+** qshrn_z5_z0_3:
+** sqshrn z5\.b, {z0\.h - z1\.h}, #3
+** ret
+*/
+TEST_X2_NARROW (qshrn_z5_z0_3, svint16x2_t, svint8_t,
+ z5 = svqshrn_n_s8_s16_x2 (z0, 3),
+ z5 = svqshrn_s8 (z0, 3))
+
+/*
+** qshrn_z22_z16_7:
+** sqshrn z22\.b, {z16\.h - z17\.h}, #7
+** ret
+*/
+TEST_X2_NARROW (qshrn_z22_z16_7, svint16x2_t, svint8_t,
+ z22 = svqshrn_n_s8_s16_x2 (z16, 7),
+ z22 = svqshrn_s8 (z16, 7))
diff --git a/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qshrn_u16_x2.c b/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qshrn_u16_x2.c
new file mode 100644
index 00000000000..a0e6df45ca6
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qshrn_u16_x2.c
@@ -0,0 +1,53 @@
+/* { dg-do assemble { target aarch64_asm_sme2p3_ok } } */
+/* { dg-do compile { target { ! aarch64_asm_sme2p3_ok } } } */
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sme2_acle.h"
+#pragma GCC target "+sme2p3"
+
+/*
+** qshrn_z0_z0_1:
+** uqshrn z0\.h, {z0\.s - z1\.s}, #1
+** ret
+*/
+TEST_X2_NARROW (qshrn_z0_z0_1, svuint32x2_t, svuint16_t,
+ z0_res = svqshrn_n_u16_u32_x2 (z0, 1),
+ z0_res = svqshrn_u16 (z0, 1))
+
+/*
+** qshrn_z0_z6_16:
+** uqshrn z0\.h, {z6\.s - z7\.s}, #16
+** ret
+*/
+TEST_X2_NARROW (qshrn_z0_z6_16, svuint32x2_t, svuint16_t,
+ z0_res = svqshrn_n_u16_u32_x2 (z6, 16),
+ z0_res = svqshrn_u16 (z6, 16))
+
+/*
+** qshrn_z0_z29_13:
+** mov [^\n]+
+** mov [^\n]+
+** uqshrn z0\.h, [^\n]+, #13
+** ret
+*/
+TEST_X2_NARROW (qshrn_z0_z29_13, svuint32x2_t, svuint16_t,
+ z0_res = svqshrn_n_u16_u32_x2 (z29, 13),
+ z0_res = svqshrn_u16 (z29, 13))
+
+/*
+** qshrn_z5_z0_11:
+** uqshrn z5\.h, {z0\.s - z1\.s}, #11
+** ret
+*/
+TEST_X2_NARROW (qshrn_z5_z0_11, svuint32x2_t, svuint16_t,
+ z5 = svqshrn_n_u16_u32_x2 (z0, 11),
+ z5 = svqshrn_u16 (z0, 11))
+
+/*
+** qshrn_z22_z16_15:
+** uqshrn z22\.h, {z16\.s - z17\.s}, #15
+** ret
+*/
+TEST_X2_NARROW (qshrn_z22_z16_15, svuint32x2_t, svuint16_t,
+ z22 = svqshrn_n_u16_u32_x2 (z16, 15),
+ z22 = svqshrn_u16 (z16, 15))
diff --git a/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qshrn_u8_x2.c b/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qshrn_u8_x2.c
new file mode 100644
index 00000000000..f74e76c9f9d
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qshrn_u8_x2.c
@@ -0,0 +1,53 @@
+/* { dg-do assemble { target aarch64_asm_sme2p3_ok } } */
+/* { dg-do compile { target { ! aarch64_asm_sme2p3_ok } } } */
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sme2_acle.h"
+#pragma GCC target "+sme2p3"
+
+/*
+** qshrn_z0_z0_1:
+** uqshrn z0\.b, {z0\.h - z1\.h}, #1
+** ret
+*/
+TEST_X2_NARROW (qshrn_z0_z0_1, svuint16x2_t, svuint8_t,
+ z0_res = svqshrn_n_u8_u16_x2 (z0, 1),
+ z0_res = svqshrn_u8 (z0, 1))
+
+/*
+** qshrn_z0_z6_8:
+** uqshrn z0\.b, {z6\.h - z7\.h}, #8
+** ret
+*/
+TEST_X2_NARROW (qshrn_z0_z6_8, svuint16x2_t, svuint8_t,
+ z0_res = svqshrn_n_u8_u16_x2 (z6, 8),
+ z0_res = svqshrn_u8 (z6, 8))
+
+/*
+** qshrn_z0_z29_5:
+** mov [^\n]+
+** mov [^\n]+
+** uqshrn z0\.b, [^\n]+, #5
+** ret
+*/
+TEST_X2_NARROW (qshrn_z0_z29_5, svuint16x2_t, svuint8_t,
+ z0_res = svqshrn_n_u8_u16_x2 (z29, 5),
+ z0_res = svqshrn_u8 (z29, 5))
+
+/*
+** qshrn_z5_z0_3:
+** uqshrn z5\.b, {z0\.h - z1\.h}, #3
+** ret
+*/
+TEST_X2_NARROW (qshrn_z5_z0_3, svuint16x2_t, svuint8_t,
+ z5 = svqshrn_n_u8_u16_x2 (z0, 3),
+ z5 = svqshrn_u8 (z0, 3))
+
+/*
+** qshrn_z22_z16_7:
+** uqshrn z22\.b, {z16\.h - z17\.h}, #7
+** ret
+*/
+TEST_X2_NARROW (qshrn_z22_z16_7, svuint16x2_t, svuint8_t,
+ z22 = svqshrn_n_u8_u16_x2 (z16, 7),
+ z22 = svqshrn_u8 (z16, 7))
diff --git a/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qshrun_u16_x2.c b/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qshrun_u16_x2.c
new file mode 100644
index 00000000000..72e0ff3419c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qshrun_u16_x2.c
@@ -0,0 +1,53 @@
+/* { dg-do assemble { target aarch64_asm_sme2p3_ok } } */
+/* { dg-do compile { target { ! aarch64_asm_sme2p3_ok } } } */
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sme2_acle.h"
+#pragma GCC target "+sme2p3"
+
+/*
+** qshrn_z0_z0_1:
+** sqshrun z0\.h, {z0\.s - z1\.s}, #1
+** ret
+*/
+TEST_X2_NARROW (qshrn_z0_z0_1, svint32x2_t, svuint16_t,
+ z0_res = svqshrun_n_u16_s32_x2 (z0, 1),
+ z0_res = svqshrun_u16 (z0, 1))
+
+/*
+** qshrn_z0_z6_16:
+** sqshrun z0\.h, {z6\.s - z7\.s}, #16
+** ret
+*/
+TEST_X2_NARROW (qshrn_z0_z6_16, svint32x2_t, svuint16_t,
+ z0_res = svqshrun_n_u16_s32_x2 (z6, 16),
+ z0_res = svqshrun_u16 (z6, 16))
+
+/*
+** qshrn_z0_z29_13:
+** mov [^\n]+
+** mov [^\n]+
+** sqshrun z0\.h, [^\n]+, #13
+** ret
+*/
+TEST_X2_NARROW (qshrn_z0_z29_13, svint32x2_t, svuint16_t,
+ z0_res = svqshrun_n_u16_s32_x2 (z29, 13),
+ z0_res = svqshrun_u16 (z29, 13))
+
+/*
+** qshrn_z5_z0_11:
+** sqshrun z5\.h, {z0\.s - z1\.s}, #11
+** ret
+*/
+TEST_X2_NARROW (qshrn_z5_z0_11, svint32x2_t, svuint16_t,
+ z5 = svqshrun_n_u16_s32_x2 (z0, 11),
+ z5 = svqshrun_u16 (z0, 11))
+
+/*
+** qshrn_z22_z16_15:
+** sqshrun z22\.h, {z16\.s - z17\.s}, #15
+** ret
+*/
+TEST_X2_NARROW (qshrn_z22_z16_15, svint32x2_t, svuint16_t,
+ z22 = svqshrun_n_u16_s32_x2 (z16, 15),
+ z22 = svqshrun_u16 (z16, 15))
diff --git a/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qshrun_u8_x2.c b/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qshrun_u8_x2.c
new file mode 100644
index 00000000000..74255dcb137
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sme2/acle-asm/qshrun_u8_x2.c
@@ -0,0 +1,53 @@
+/* { dg-do assemble { target aarch64_asm_sme2p3_ok } } */
+/* { dg-do compile { target { ! aarch64_asm_sme2p3_ok } } } */
+/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
+
+#include "test_sme2_acle.h"
+#pragma GCC target "+sme2p3"
+
+/*
+** qshrn_z0_z0_1:
+** sqshrun z0\.b, {z0\.h - z1\.h}, #1
+** ret
+*/
+TEST_X2_NARROW (qshrn_z0_z0_1, svint16x2_t, svuint8_t,
+ z0_res = svqshrun_n_u8_s16_x2 (z0, 1),
+ z0_res = svqshrun_u8 (z0, 1))
+
+/*
+** qshrn_z0_z6_8:
+** sqshrun z0\.b, {z6\.h - z7\.h}, #8
+** ret
+*/
+TEST_X2_NARROW (qshrn_z0_z6_8, svint16x2_t, svuint8_t,
+ z0_res = svqshrun_n_u8_s16_x2 (z6, 8),
+ z0_res = svqshrun_u8 (z6, 8))
+
+/*
+** qshrn_z0_z29_5:
+** mov [^\n]+
+** mov [^\n]+
+** sqshrun z0\.b, [^\n]+, #5
+** ret
+*/
+TEST_X2_NARROW (qshrn_z0_z29_5, svint16x2_t, svuint8_t,
+ z0_res = svqshrun_n_u8_s16_x2 (z29, 5),
+ z0_res = svqshrun_u8 (z29, 5))
+
+/*
+** qshrn_z5_z0_3:
+** sqshrun z5\.b, {z0\.h - z1\.h}, #3
+** ret
+*/
+TEST_X2_NARROW (qshrn_z5_z0_3, svint16x2_t, svuint8_t,
+ z5 = svqshrun_n_u8_s16_x2 (z0, 3),
+ z5 = svqshrun_u8 (z0, 3))
+
+/*
+** qshrn_z22_z16_7:
+** sqshrun z22\.b, {z16\.h - z17\.h}, #7
+** ret
+*/
+TEST_X2_NARROW (qshrn_z22_z16_7, svint16x2_t, svuint8_t,
+ z22 = svqshrun_n_u8_s16_x2 (z16, 7),
+ z22 = svqshrun_u8 (z16, 7))
--
2.43.0