Re: [PATCH v3] aarch64: Add SVE AES2 ACLE builtins
Alice Carlotti <[email protected]>
| Newsgroups | gmane.comp.gcc.patches |
|---|---|
| Message-ID | <[email protected]> |
On Wed, Aug 05, 2026 at 10:35:35AM +0000, Sivan Shani wrote:
> Add ACLE support for the SVE AES2 multi-vector AES indexed
> instructions and the 128-bit PMULL/PMLAL pair forms.
>
> This adds builtin shapes and expanders for svaese_lane, svaesd_lane,
> svaesemc_lane, svaesdimc_lane, svpmull_pair and svpmlal_pair, together
> with the corresponding RTL patterns and tests. Also add preprocessor
> feature macros for SVE AES2 and SSVE AES.
Thanks, this looks good now - I can commit this for you later today.
>
> gcc/
> * config/aarch64/aarch64-c.cc (aarch64_update_cpp_builtins): Define
> __ARM_FEATURE_SVE_AES2 and __ARM_FEATURE_SSVE_AES.
> * config/aarch64/aarch64.h (TARGET_SVE_AES2): Define.
> * config/aarch64/aarch64.md (UNSPEC_SSVE_LANE_SELECT): New unspec.
> * config/aarch64/aarch64-sve-builtins-functions.h
> (unspec_based_aes_lane_function): New typedef.
> (unspec_based_aes_lane_mc_function): Likewise.
> * config/aarch64/aarch64-sve-builtins-shapes.cc
> (binary_to_pair_opt_n_def): New shape.
> (ternary_to_pair_opt_n_def): Likewise.
> (binary_aes_lane_def): Likewise.
> * config/aarch64/aarch64-sve-builtins-shapes.h: Declare new shapes.
> * config/aarch64/aarch64-sve-builtins-sve2.cc: Add new function
> entries.
> * config/aarch64/aarch64-sve-builtins-sve2.def: Add new ACLE
> builtin definitions.
> * config/aarch64/aarch64-sve-builtins-sve2.h: Declare new functions.
> * config/aarch64/aarch64-sve2.md: Add PMULL/PMLAL pair and AES
> indexed multi-vector patterns.
> * config/aarch64/iterators.md: Add iterators and attrs for new
> patterns.
> * config/aarch64/predicates.md (const_0_to_3_operand): New predicate.
I didn't notice this before, but this part of the changelog seems too brief. I
plan to commit it with the following here instead:
* config/aarch64/aarch64-sve-builtins-shapes.h
(binary_aes_lane): Declare.
(binary_to_pair_opt_n): Likewise.
(ternary_to_pair_opt_n): Likewise.
* config/aarch64/aarch64-sve-builtins-sve2.cc
(svaesd_lane, svaesdimc_lane): New functions.
(svaese_lane, svaesemc_lane): Likewise.
(svpmlal_pair, svpmull_pair): Likewise.
* config/aarch64/aarch64-sve-builtins-sve2.def: Add the SVE_AES2
intrinsics.
* config/aarch64/aarch64-sve-builtins-sve2.h
(svaesd_lane, svaesdimc_lane): Declare.
(svaese_lane, svaesemc_lane): Likewise.
(svpmlal_pair, svpmull_pair): Likewise.
* config/aarch64/aarch64-sve2.md: Add PMULL/PMLAL pair and AES
indexed multi-vector patterns.
* config/aarch64/iterators.md (SVE_QIx24): New mode iterator.
(UNSPEC_PMULL_PAIR, UNSPEC_PMLAL_PAIR): New unspecs.
(aes_fused_op, aes_mc_unspec): New int iterators.
* config/aarch64/predicates.md (const_0_to_3_operand): New predicate.
Alice
>
> gcc/testsuite/
> * gcc.target/aarch64/pragma_cpp_predefs_5.c: Test new feature macros.
> * gcc.target/aarch64/sve/acle/asm/test_sve_acle.h
> (TEST_XN_INDEXED): New macro.
> * gcc.target/aarch64/sve2/acle/asm/aesd_lane_u8.c: New test.
> * gcc.target/aarch64/sve2/acle/asm/aesdimc_lane_u8.c: New test.
> * gcc.target/aarch64/sve2/acle/asm/aese_lane_u8.c: New test.
> * gcc.target/aarch64/sve2/acle/asm/aesemc_lane_u8.c: New test.
> * gcc.target/aarch64/sve2/acle/asm/pmlal_pair_u64.c: New test.
> * gcc.target/aarch64/sve2/acle/asm/pmull_pair_u64.c: New test.
> * lib/target-supports.exp: Add sve-aes2 assembler probe.
> ---
>
> Thanks for the review.
> The unused change to TEST_XN_SINGLE has been removed.
> Some small changes were needed for rebasing on master.
>
>
> gcc/config/aarch64/aarch64-acle-builtins.cc | 18 ++-
> gcc/config/aarch64/aarch64-acle-builtins.h | 2 +
> gcc/config/aarch64/aarch64-c.cc | 4 +
> .../aarch64/aarch64-sve-builtins-functions.h | 5 +
> .../aarch64/aarch64-sve-builtins-shapes.cc | 116 ++++++++++++++++++
> .../aarch64/aarch64-sve-builtins-shapes.h | 3 +
> .../aarch64/aarch64-sve-builtins-sve2.cc | 16 ++-
> .../aarch64/aarch64-sve-builtins-sve2.def | 10 ++
> .../aarch64/aarch64-sve-builtins-sve2.h | 7 ++
> gcc/config/aarch64/aarch64-sve2.md | 71 +++++++++++
> gcc/config/aarch64/aarch64.h | 2 +
> gcc/config/aarch64/aarch64.md | 3 +
> gcc/config/aarch64/iterators.md | 8 ++
> gcc/config/aarch64/predicates.md | 4 +
> .../gcc.target/aarch64/pragma_cpp_predefs_5.c | 22 ++++
> .../aarch64/sve/acle/asm/test_sve_acle.h | 18 +++
> .../aarch64/sve2/acle/asm/aesd_lane_u8.c | 92 ++++++++++++++
> .../aarch64/sve2/acle/asm/aesdimc_lane_u8.c | 91 ++++++++++++++
> .../aarch64/sve2/acle/asm/aese_lane_u8.c | 91 ++++++++++++++
> .../aarch64/sve2/acle/asm/aesemc_lane_u8.c | 91 ++++++++++++++
> .../aarch64/sve2/acle/asm/pmlal_pair_u64.c | 113 +++++++++++++++++
> .../aarch64/sve2/acle/asm/pmull_pair_u64.c | 65 ++++++++++
> gcc/testsuite/lib/target-supports.exp | 4 +-
> 23 files changed, 850 insertions(+), 6 deletions(-)
> create mode 100644 gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aesd_lane_u8.c
> create mode 100644 gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aesdimc_lane_u8.c
> create mode 100644 gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aese_lane_u8.c
> create mode 100644 gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aesemc_lane_u8.c
> create mode 100644 gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/pmlal_pair_u64.c
> create mode 100644 gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/pmull_pair_u64.c
>
> diff --git a/gcc/config/aarch64/aarch64-acle-builtins.cc b/gcc/config/aarch64/aarch64-acle-builtins.cc
> index e1187859ec5..98ba6c8d347 100644
> --- a/gcc/config/aarch64/aarch64-acle-builtins.cc
> +++ b/gcc/config/aarch64/aarch64-acle-builtins.cc
> @@ -2326,8 +2326,22 @@ finish_opt_n_resolution (unsigned int argno, unsigned int first_argno,
> unsigned int expected_bits,
> type_suffix_index inferred_type)
> {
> - if (inferred_type == NUM_TYPE_SUFFIXES)
> - inferred_type = first_type;
> + return finish_opt_n_resolution (argno, first_argno, first_type,
> + expected_tclass, expected_bits,
> + (inferred_type == NUM_TYPE_SUFFIXES
> + ? sve_type (first_type)
> + : sve_type (inferred_type)));
> +}
> +
> +tree function_resolver::
> +finish_opt_n_resolution (unsigned int argno, unsigned int first_argno,
> + type_suffix_index first_type,
> + type_class_index expected_tclass,
> + unsigned int expected_bits,
> + sve_type inferred_type)
> +{
> + if (!inferred_type)
> + inferred_type = sve_type (first_type);
> tree scalar_form = lookup_form (MODE_n, inferred_type);
>
> /* Allow the final argument to be scalar, if an _n form exists. */
> diff --git a/gcc/config/aarch64/aarch64-acle-builtins.h b/gcc/config/aarch64/aarch64-acle-builtins.h
> index 6314ec5d312..42150a2632b 100644
> --- a/gcc/config/aarch64/aarch64-acle-builtins.h
> +++ b/gcc/config/aarch64/aarch64-acle-builtins.h
> @@ -590,6 +590,8 @@ public:
> type_class_index = SAME_TYPE_CLASS,
> unsigned int = SAME_SIZE,
> type_suffix_index = NUM_TYPE_SUFFIXES);
> + tree finish_opt_n_resolution (unsigned int, unsigned int, type_suffix_index,
> + type_class_index, unsigned int, sve_type);
> tree finish_opt_single_resolution (unsigned int, unsigned int, sve_type,
> type_class_index = SAME_TYPE_CLASS);
>
> diff --git a/gcc/config/aarch64/aarch64-c.cc b/gcc/config/aarch64/aarch64-c.cc
> index eb47a57ede4..454c318d0d8 100644
> --- a/gcc/config/aarch64/aarch64-c.cc
> +++ b/gcc/config/aarch64/aarch64-c.cc
> @@ -227,6 +227,10 @@ aarch64_update_cpp_builtins (cpp_reader *pfile)
> aarch64_def_or_undef (AARCH64_HAVE_ISA (SVE2), "__ARM_FEATURE_SVE2", pfile);
> aarch64_def_or_undef (AARCH64_HAVE_ISA (SVE2) && AARCH64_HAVE_ISA (SVE_AES),
> "__ARM_FEATURE_SVE2_AES", pfile);
> + aarch64_def_or_undef (AARCH64_HAVE_ISA (SVE_AES2), "__ARM_FEATURE_SVE_AES2",
> + pfile);
> + aarch64_def_or_undef (AARCH64_HAVE_ISA (SSVE_AES), "__ARM_FEATURE_SSVE_AES",
> + pfile);
> aarch64_def_or_undef (AARCH64_HAVE_ISA (SVE_BITPERM)
> && AARCH64_HAVE_ISA (SVE2),
> "__ARM_FEATURE_SVE2_BITPERM", pfile);
> diff --git a/gcc/config/aarch64/aarch64-sve-builtins-functions.h b/gcc/config/aarch64/aarch64-sve-builtins-functions.h
> index 072c0bf2c89..ebbefaf739e 100644
> --- a/gcc/config/aarch64/aarch64-sve-builtins-functions.h
> +++ b/gcc/config/aarch64/aarch64-sve-builtins-functions.h
> @@ -397,6 +397,11 @@ typedef unspec_based_function_exact_insn<code_for_aarch64_sve_sub>
> typedef unspec_based_function_exact_insn<code_for_aarch64_sve_sub_lane>
> unspec_based_sub_lane_function;
>
> +typedef unspec_based_function_exact_insn<code_for_aarch64_sve2_aes_lane>
> + unspec_based_aes_lane_function;
> +typedef unspec_based_function_exact_insn<code_for_aarch64_sve2_aes_lane_mc>
> + unspec_based_aes_lane_mc_function;
> +
> /* A function that has conditional and unconditional forms, with both
> forms being associated with a single unspec each. */
> class cond_or_uncond_unspec_function : public function_base
> diff --git a/gcc/config/aarch64/aarch64-sve-builtins-shapes.cc b/gcc/config/aarch64/aarch64-sve-builtins-shapes.cc
> index 90bc80d8195..cf128bc9464 100644
> --- a/gcc/config/aarch64/aarch64-sve-builtins-shapes.cc
> +++ b/gcc/config/aarch64/aarch64-sve-builtins-shapes.cc
> @@ -1522,6 +1522,122 @@ struct binary_int_opt_single_n_def : public overloaded_base<0>
> };
> SHAPE (binary_int_opt_single_n);
>
> +/* sv<t0>x<g>_t svfoo[_t0_g](sv<t0>_t, sv<t0>_t)
> + sv<t0>x<g>_t svfoo[_n_t0_g](sv<t0>_t, <t0>_t). */
> +struct binary_to_pair_opt_n_def : public overloaded_base<0>
> +{
> + bool explicit_group_suffix_p () const override { return false; }
> +
> + void
> + build (function_builder &b, const function_group_info &group) const override
> + {
> + b.add_overloaded_functions (group, MODE_none);
> + build_all (b, "t0,v0,v0", group, MODE_none);
> + build_all (b, "t0,v0,s0", group, MODE_n);
> + }
> +
> + tree
> + resolve (function_resolver &r) const override
> + {
> + unsigned int i, nargs;
> + type_suffix_index type;
> + if (!r.check_gp_argument (2, i, nargs)
> + || (type = r.infer_vector_type (i)) == NUM_TYPE_SUFFIXES)
> + return error_mark_node;
> +
> + return r.finish_opt_n_resolution (i + 1, i, type, r.SAME_TYPE_CLASS,
> + r.SAME_SIZE, sve_type (type, 2));
> + }
> +};
> +SHAPE (binary_to_pair_opt_n);
> +
> +/* sv<t0>x<g>_t svfoo[_t0_g](sv<t0>x<g>_t, sv<t0>_t, sv<t0>_t)
> + sv<t0>x<g>_t svfoo[_n_t0_g](sv<t0>x<g>_t, sv<t0>_t, <t0>_t). */
> +struct ternary_to_pair_opt_n_def : public overloaded_base<0>
> +{
> + bool explicit_group_suffix_p () const override { return false; }
> +
> + void
> + build (function_builder &b, const function_group_info &group) const override
> + {
> + b.add_overloaded_functions (group, MODE_none);
> + build_all (b, "t0,t0,v0,v0", group, MODE_none);
> + build_all (b, "t0,t0,v0,s0", group, MODE_n);
> + }
> +
> + tree
> + resolve (function_resolver &r) const override
> + {
> + unsigned int i, nargs;
> + sve_type type;
> + if (!r.check_gp_argument (3, i, nargs)
> + || !(type = r.infer_sve_type (i)))
> + return error_mark_node;
> +
> + if (type.num_vectors != 2)
> + {
> + r.report_incorrect_num_vectors (i, type, 2);
> + return error_mark_node;
> + }
> +
> + if (!r.require_derived_vector_type (i + 1, i, type))
> + return error_mark_node;
> +
> + return r.finish_opt_n_resolution (i + 2, i + 1, type.type,
> + r.SAME_TYPE_CLASS, r.SAME_SIZE, type);
> + }
> +};
> +SHAPE (ternary_to_pair_opt_n);
> +
> +/* svuint8x2_t svaes<...>_lane[_u8_x2] (svuint8x2_t zdn, svuint8_t zm, uint64_t
> + index);
> + and
> + svuint8x4_t svaes<...>_lane[_u8_x4] (svuint8x4_t zdn, svuint8_t zm, uint64_t
> + index);
> + When index is in range[0-3]
> +*/
> +struct binary_aes_lane_def : public overloaded_base<0>
> +{
> + bool explicit_group_suffix_p () const override { return false; }
> +
> + void
> + build (function_builder &b, const function_group_info &group) const override
> + {
> + b.add_overloaded_functions (group, MODE_none);
> + build_all (b, "t0,t0,v0,su64", group, MODE_none);
> + }
> +
> + tree
> + resolve (function_resolver &r) const override
> + {
> + if (!r.check_num_arguments (3))
> + return error_mark_node;
> +
> + sve_type type = r.infer_sve_type (0);
> + if (!type)
> + return error_mark_node;
> +
> + if (type.num_vectors != 2 && type.num_vectors != 4)
> + return error_mark_node;
> +
> + if (!r.require_vector_type (1, VECTOR_TYPE_svuint8_t))
> + return error_mark_node;
> +
> + if (!r.require_integer_immediate (2))
> + return error_mark_node;
> +
> + return r.resolve_to (MODE_none, type);
> + }
> +
> + bool
> + check (function_checker &c) const override
> + {
> + return c.require_immediate_lane_index (2, 0, 4);
> + }
> +};
> +SHAPE (binary_aes_lane);
> +
> +
> /* sv<t0>_t svfoo_<t0>(sv<t0>_t, sv<t0>_t, uint64_t)
>
> where the final argument is an integer constant expression in the
> diff --git a/gcc/config/aarch64/aarch64-sve-builtins-shapes.h b/gcc/config/aarch64/aarch64-sve-builtins-shapes.h
> index ce02b5a45ec..0fca00b36d1 100644
> --- a/gcc/config/aarch64/aarch64-sve-builtins-shapes.h
> +++ b/gcc/config/aarch64/aarch64-sve-builtins-shapes.h
> @@ -81,6 +81,7 @@ namespace aarch64_acle
> extern const function_shape *const adr_index;
> extern const function_shape *const adr_offset;
> extern const function_shape *const binary;
> + extern const function_shape *const binary_aes_lane;
> extern const function_shape *const binary_int_opt_n;
> extern const function_shape *const binary_int_opt_single_n;
> extern const function_shape *const binary_lane;
> @@ -95,6 +96,7 @@ namespace aarch64_acle
> extern const function_shape *const binary_rotate;
> extern const function_shape *const binary_scalar;
> extern const function_shape *const binary_single;
> + extern const function_shape *const binary_to_pair_opt_n;
> extern const function_shape *const binary_to_uint;
> extern const function_shape *const binary_uint;
> extern const function_shape *const binary_uint_n;
> @@ -233,6 +235,7 @@ namespace aarch64_acle
> extern const function_shape *const ternary_rotate;
> extern const function_shape *const ternary_shift_left_imm;
> extern const function_shape *const ternary_shift_right_imm;
> + extern const function_shape *const ternary_to_pair_opt_n;
> extern const function_shape *const ternary_uint;
> extern const function_shape *const ternary_uintq_intq;
> extern const function_shape *const ternary_uintq_intq_lane;
> diff --git a/gcc/config/aarch64/aarch64-sve-builtins-sve2.cc b/gcc/config/aarch64/aarch64-sve-builtins-sve2.cc
> index 738ef201324..c22b5b176be 100644
> --- a/gcc/config/aarch64/aarch64-sve-builtins-sve2.cc
> +++ b/gcc/config/aarch64/aarch64-sve-builtins-sve2.cc
> @@ -1070,7 +1070,15 @@ FUNCTION (svaddqv, reduction, (UNSPEC_ADDQV, UNSPEC_ADDQV, UNSPEC_FADDQV))
> FUNCTION (svaddwb, unspec_based_function, (UNSPEC_SADDWB, UNSPEC_UADDWB))
> FUNCTION (svaddwt, unspec_based_function, (UNSPEC_SADDWT, UNSPEC_UADDWT))
> FUNCTION (svaesd, fixed_insn_function, (CODE_FOR_aarch64_sve2_aesd))
> +FUNCTION (svaesd_lane, unspec_based_aes_lane_function,
> + (UNSPEC_NONE, UNSPEC_AESD))
> +FUNCTION (svaesdimc_lane, unspec_based_aes_lane_mc_function,
> + (UNSPEC_NONE, UNSPEC_AESD))
> FUNCTION (svaese, fixed_insn_function, (CODE_FOR_aarch64_sve2_aese))
> +FUNCTION (svaese_lane, unspec_based_aes_lane_function,
> + (UNSPEC_NONE, UNSPEC_AESE))
> +FUNCTION (svaesemc_lane, unspec_based_aes_lane_mc_function,
> + (UNSPEC_NONE, UNSPEC_AESE))
> FUNCTION (svaesimc, fixed_insn_function, (CODE_FOR_aarch64_sve2_aesimc))
> FUNCTION (svaesmc, fixed_insn_function, (CODE_FOR_aarch64_sve2_aesmc))
> FUNCTION (svamax, faminmaximpl, (UNSPEC_COND_FAMAX, UNSPEC_FAMAX))
> @@ -1194,13 +1202,17 @@ FUNCTION (svnbsl, CODE_FOR_MODE0 (aarch64_sve2_nbsl),)
> FUNCTION (svnmatch, svmatch_svnmatch_impl, (UNSPEC_NMATCH))
> FUNCTION (svorqv, reduction, (UNSPEC_ORQV, UNSPEC_ORQV))
> FUNCTION (svpext_lane, svpext_lane_impl,)
> +FUNCTION (svpmlal_pair, fixed_insn_function, (CODE_FOR_aarch64_sve_pmlal_pair))
> FUNCTION (svpmov, svpmov_impl,)
> FUNCTION (svpmov_lane, svpmov_lane_impl,)
> FUNCTION (svpmul, CODE_FOR_MODE0 (aarch64_sve2_pmul),)
> +FUNCTION (svpmull_pair, fixed_insn_function, (CODE_FOR_aarch64_sve_pmull_pair))
> FUNCTION (svpmullb, unspec_based_function, (UNSPEC_NONE, UNSPEC_PMULLB))
> -FUNCTION (svpmullb_pair, unspec_based_function, (UNSPEC_NONE, UNSPEC_PMULLB_PAIR))
> +FUNCTION (svpmullb_pair, unspec_based_function,
> + (UNSPEC_NONE, UNSPEC_PMULLB_PAIR))
> FUNCTION (svpmullt, unspec_based_function, (UNSPEC_NONE, UNSPEC_PMULLT))
> -FUNCTION (svpmullt_pair, unspec_based_function, (UNSPEC_NONE, UNSPEC_PMULLT_PAIR))
> +FUNCTION (svpmullt_pair, unspec_based_function,
> + (UNSPEC_NONE, UNSPEC_PMULLT_PAIR))
> FUNCTION (svpsel_lane, svpsel_lane_impl,)
> FUNCTION (svqabs, rtx_code_function, (SS_ABS, UNKNOWN))
> FUNCTION (svqcadd, svqcadd_impl,)
> diff --git a/gcc/config/aarch64/aarch64-sve-builtins-sve2.def b/gcc/config/aarch64/aarch64-sve-builtins-sve2.def
> index a977b2c1c1a..87f7b472dd9 100644
> --- a/gcc/config/aarch64/aarch64-sve-builtins-sve2.def
> +++ b/gcc/config/aarch64/aarch64-sve-builtins-sve2.def
> @@ -206,6 +206,16 @@ DEF_SVE_FUNCTION (svpmullb_pair, binary_opt_n, d_unsigned, none)
> DEF_SVE_FUNCTION (svpmullt_pair, binary_opt_n, d_unsigned, none)
> #undef REQUIRED_EXTENSIONS
>
> +#define REQUIRED_EXTENSIONS streaming_compatible (AARCH64_FL_SVE_AES2, \
> + AARCH64_FL_SSVE_AES)
> +DEF_SVE_FUNCTION_GS (svaesd_lane, binary_aes_lane, b_unsigned, x24, none)
> +DEF_SVE_FUNCTION_GS (svaese_lane, binary_aes_lane, b_unsigned, x24, none)
> +DEF_SVE_FUNCTION_GS (svaesdimc_lane, binary_aes_lane, b_unsigned, x24, none)
> +DEF_SVE_FUNCTION_GS (svaesemc_lane, binary_aes_lane, b_unsigned, x24, none)
> +DEF_SVE_FUNCTION_GS (svpmull_pair, binary_to_pair_opt_n, d_unsigned, x2, none)
> +DEF_SVE_FUNCTION_GS (svpmlal_pair, ternary_to_pair_opt_n, d_unsigned, x2, none)
> +#undef REQUIRED_EXTENSIONS
> +
> #define REQUIRED_EXTENSIONS streaming_compatible (AARCH64_FL_SVE2 \
> | AARCH64_FL_SVE_BITPERM, \
> AARCH64_FL_SSVE_BITPERM)
> diff --git a/gcc/config/aarch64/aarch64-sve-builtins-sve2.h b/gcc/config/aarch64/aarch64-sve-builtins-sve2.h
> index 4bf8f21776a..0cde240403c 100644
> --- a/gcc/config/aarch64/aarch64-sve-builtins-sve2.h
> +++ b/gcc/config/aarch64/aarch64-sve-builtins-sve2.h
> @@ -42,7 +42,11 @@ namespace aarch64_acle
> extern const function_base *const svaddwb;
> extern const function_base *const svaddwt;
> extern const function_base *const svaesd;
> + extern const function_base *const svaesd_lane;
> + extern const function_base *const svaesdimc_lane;
> extern const function_base *const svaese;
> + extern const function_base *const svaese_lane;
> + extern const function_base *const svaesemc_lane;
> extern const function_base *const svaesimc;
> extern const function_base *const svaesmc;
> extern const function_base *const svandqv;
> @@ -138,13 +142,16 @@ namespace aarch64_acle
> extern const function_base *const svnmatch;
> extern const function_base *const svorqv;
> extern const function_base *const svpext_lane;
> + extern const function_base *const svpmlal_pair;
> extern const function_base *const svpmov;
> extern const function_base *const svpmov_lane;
> extern const function_base *const svpmul;
> + extern const function_base *const svpmull_pair;
> extern const function_base *const svpmullb;
> extern const function_base *const svpmullb_pair;
> extern const function_base *const svpmullt;
> extern const function_base *const svpmullt_pair;
> + extern const function_base *const svpmlal_pair;
> extern const function_base *const svpsel_lane;
> extern const function_base *const svqabs;
> extern const function_base *const svqcadd;
> diff --git a/gcc/config/aarch64/aarch64-sve2.md b/gcc/config/aarch64/aarch64-sve2.md
> index a8bee6b6dc7..33b39412e36 100644
> --- a/gcc/config/aarch64/aarch64-sve2.md
> +++ b/gcc/config/aarch64/aarch64-sve2.md
> @@ -4225,6 +4225,35 @@ (define_insn "@aarch64_sve_<optab><mode>"
> [(set_attr "sve_type" "sve_int_pmul")]
> )
>
> +;; Polynomial multiply corresponding D elements and return the widened Q
> +;; results as a pair of consecutive destination vectors.
> +;; PMULL { <Zd1>.Q-<Zd2>.Q }, <Zn>.D, <Zm>.D
> +(define_insn "aarch64_sve_pmull_pair"
> + [(set (match_operand:VNx4DI 0 "aligned_register_operand" "=Uw2")
> + (unspec:VNx4DI
> + [(match_operand:VNx2DI 1 "register_operand" "w")
> + (match_operand:VNx2DI 2 "register_operand" "w")]
> + UNSPEC_PMULL_PAIR))]
> + "TARGET_SVE_AES2"
> + "pmull\t{%S0.q - %T0.q}, %1.d, %2.d"
> + [(set_attr "sve_type" "sve_int_pmul")]
> +)
> +
> +;; Polynomial multiply corresponding D elements and XOR-accumulate the widened
> +;; Q results into a pair of consecutive destination vectors.
> +;; PMLAL { <Zda1>.Q-<Zda2>.Q }, <Zn>.D, <Zm>.D
> +(define_insn "aarch64_sve_pmlal_pair"
> + [(set (match_operand:VNx4DI 0 "aligned_register_operand" "=Uw2")
> + (unspec:VNx4DI
> + [(match_operand:VNx4DI 1 "aligned_register_operand" "0")
> + (match_operand:VNx2DI 2 "register_operand" "w")
> + (match_operand:VNx2DI 3 "register_operand" "w")]
> + UNSPEC_PMLAL_PAIR))]
> + "TARGET_SVE_AES2"
> + "pmlal\t{%S0.q - %T0.q}, %2.d, %3.d"
> + [(set_attr "sve_type" "sve_int_pmul")]
> +)
> +
> ;; =========================================================================
> ;; == Comparisons and selects
> ;; =========================================================================
> @@ -4830,6 +4859,10 @@ (define_insn "@aarch64_sve_luti<LUTI_BITS><mode>"
> ;; - AESE
> ;; - AESIMC
> ;; - AESMC
> +;; - AESD (indexed, two registers and four registers)
> +;; - AESE (indexed, two registers and four registers)
> +;; - AESEMC (indexed, two registers and four registers)
> +;; - AESDIMC (indexed, two registers and four registers)
> ;; -------------------------------------------------------------------------
>
> ;; AESD and AESE.
> @@ -4891,6 +4924,44 @@ (define_insn "*aarch64_sve2_aesd_fused"
> (set_attr "length" "8")]
> )
>
> +;; AESE and AESD, indexed, two registers and four registers.
> +;; AES<E/D> { <Zdn1>.B-<Zdn(2/4)>.B }, { <Zdn1>.B-<Zdn(2/4)>.B }, <Zm>.Q[<index>]
> +
> +(define_insn "@aarch64_sve2_aes<aes_op>_lane<mode>"
> + [(set (match_operand:SVE_QIx24 0 "aligned_register_operand" "=Uw<vector_count>")
> + (unspec:SVE_QIx24
> + [(xor:SVE_QIx24
> + (match_operand:SVE_QIx24 1 "aligned_register_operand" "0")
> + (unspec:SVE_QIx24
> + [(match_operand:VNx16QI 2 "register_operand" "w")
> + (match_operand:SI 3 "const_0_to_3_operand")]
> + UNSPEC_SSVE_LANE_SELECT))]
> + CRYPTO_AES))]
> + "TARGET_SVE_AES2"
> + "aes<aes_op>\t%0, %0, %2.q[%3]"
> + [(set_attr "type" "crypto_aese")]
> +)
> +
> +;; AESEMC and AESDIMC, indexed, two registers and four registers.
> +;; AESEMC/AESDIMC { <Zdn1>.B-<Zdn (2/4)>.B }, { <Zdn1>.B-<Zdn (2/4)>.B }, <Zm>.Q[<index>]
> +
> +(define_insn "@aarch64_sve2_aes<CRYPTO_AES>_lane_mc<mode>"
> + [(set (match_operand:SVE_QIx24 0 "aligned_register_operand" "=Uw<vector_count>")
> + (unspec:SVE_QIx24
> + [(unspec:SVE_QIx24
> + [(xor:SVE_QIx24
> + (match_operand:SVE_QIx24 1 "aligned_register_operand" "0")
> + (unspec:SVE_QIx24
> + [(match_operand:VNx16QI 2 "register_operand" "w")
> + (match_operand:SI 3 "const_0_to_3_operand")]
> + UNSPEC_SSVE_LANE_SELECT))]
> + CRYPTO_AES)]
> + <aes_mc_unspec>))]
> + "TARGET_SVE_AES2"
> + "aes<aes_fused_op>\t%0, %0, %2.q[%3]"
> + [(set_attr "type" "crypto_aesmc")]
> +)
> +
> ;; -------------------------------------------------------------------------
> ;; ---- Optional SHA-3 extensions
> ;; -------------------------------------------------------------------------
> diff --git a/gcc/config/aarch64/aarch64.h b/gcc/config/aarch64/aarch64.h
> index 50577b136ac..11fb7a9481e 100644
> --- a/gcc/config/aarch64/aarch64.h
> +++ b/gcc/config/aarch64/aarch64.h
> @@ -295,6 +295,8 @@ constexpr auto AARCH64_FL_DEFAULT_ISA_MODE ATTRIBUTE_UNUSED
> && (AARCH64_HAVE_ISA (SSVE_AES) \
> || TARGET_NON_STREAMING))
>
> +#define TARGET_SVE_AES2 (TARGET_SVE_AES && AARCH64_HAVE_ISA (SVE_AES2))
> +
> /* SVE BITPERM instructions, enabled through +sve-bitperm+sve2 for non-streaming
> and +ssve-bitperm for streaming. */
> #define TARGET_SVE_BITPERM (AARCH64_HAVE_ISA (SVE_BITPERM) \
> diff --git a/gcc/config/aarch64/aarch64.md b/gcc/config/aarch64/aarch64.md
> index 168a8a471fe..9cb55602c36 100644
> --- a/gcc/config/aarch64/aarch64.md
> +++ b/gcc/config/aarch64/aarch64.md
> @@ -392,6 +392,9 @@ (define_c_enum "unspec" [
> ;; Represents an SVE-style lane index, in which the indexing applies
> ;; within the containing 128-bit block.
> UNSPEC_SVE_LANE_SELECT
> + ;; Represents an SVE-style lane index, in which the indexing applies
> + ;; within the containing 512-bit block.
> + UNSPEC_SSVE_LANE_SELECT
> UNSPEC_SVE_CNT_PAT
> UNSPEC_SVE_PREFETCH
> UNSPEC_SVE_PREFETCH_GATHER
> diff --git a/gcc/config/aarch64/iterators.md b/gcc/config/aarch64/iterators.md
> index 0d319751430..e7ae93d1896 100644
> --- a/gcc/config/aarch64/iterators.md
> +++ b/gcc/config/aarch64/iterators.md
> @@ -711,6 +711,9 @@ (define_mode_iterator SVE_SI [VNx2SI VNx4SI])
>
> (define_mode_iterator SVE_DIx24 [VNx4DI VNx8DI])
>
> +;; SVE integer vector modes with 2 and 4 vectors of 8-bit elements.
> +(define_mode_iterator SVE_QIx24 [VNx32QI VNx64QI])
> +
> ;; SVE modes with 2 or 4 elements.
> (define_mode_iterator SVE_24 [VNx2QI VNx2HI VNx2HF VNx2BF VNx2SI VNx2SF
> VNx2DI VNx2DF
> @@ -892,6 +895,8 @@ (define_c_enum "unspec"
> UNSPEC_SQDMULH ; Used in aarch64-simd.md.
> UNSPEC_SQRDMULH ; Used in aarch64-simd.md.
> UNSPEC_PMUL ; Used in aarch64-simd.md.
> + UNSPEC_PMULL_PAIR ; Used in aarch64-sve2.md.
> + UNSPEC_PMLAL_PAIR ; Used in aarch64-sve2.md.
> UNSPEC_FMULX ; Used in aarch64-simd.md.
> UNSPEC_USQADD ; Used in aarch64-simd.md.
> UNSPEC_SUQADD ; Used in aarch64-simd.md.
> @@ -3698,6 +3703,9 @@ (define_int_iterator CRC [UNSPEC_CRC32B UNSPEC_CRC32H UNSPEC_CRC32W
>
> (define_int_iterator CRYPTO_AES [UNSPEC_AESE UNSPEC_AESD])
> (define_int_iterator CRYPTO_AESMC [UNSPEC_AESMC UNSPEC_AESIMC])
> +(define_int_attr aes_fused_op [(UNSPEC_AESE "emc") (UNSPEC_AESD "dimc")])
> +(define_int_attr aes_mc_unspec [(UNSPEC_AESE "UNSPEC_AESMC")
> + (UNSPEC_AESD "UNSPEC_AESIMC")])
>
> (define_int_iterator CRYPTO_SHA1 [UNSPEC_SHA1C UNSPEC_SHA1M UNSPEC_SHA1P])
>
> diff --git a/gcc/config/aarch64/predicates.md b/gcc/config/aarch64/predicates.md
> index f02486c2d9a..e2911a0bccd 100644
> --- a/gcc/config/aarch64/predicates.md
> +++ b/gcc/config/aarch64/predicates.md
> @@ -50,6 +50,10 @@ (define_predicate "const0_to_1_operand"
> (and (match_code "const_int")
> (match_test "IN_RANGE (INTVAL (op), 0, 1)")))
>
> +(define_predicate "const_0_to_3_operand"
> + (and (match_code "const_int")
> + (match_test "IN_RANGE (INTVAL (op), 0, 3)")))
> +
> (define_predicate "const_0_to_7_operand"
> (and (match_code "const_int")
> (match_test "IN_RANGE (INTVAL (op), 0, 7)")))
> diff --git a/gcc/testsuite/gcc.target/aarch64/pragma_cpp_predefs_5.c b/gcc/testsuite/gcc.target/aarch64/pragma_cpp_predefs_5.c
> index bb83ab434b5..0bec72d70a2 100644
> --- a/gcc/testsuite/gcc.target/aarch64/pragma_cpp_predefs_5.c
> +++ b/gcc/testsuite/gcc.target/aarch64/pragma_cpp_predefs_5.c
> @@ -41,6 +41,14 @@
> #error "__ARM_FEATURE_SVE2_SHA3 is defined but should not be!"
> #endif
>
> +#ifdef __ARM_FEATURE_SSVE_AES
> +#error "__ARM_FEATURE_SSVE_AES is defined but should not be!"
> +#endif
> +
> +#ifdef __ARM_FEATURE_SVE_AES2
> +#error "__ARM_FEATURE_SVE_AES2 is defined but should not be!"
> +#endif
> +
> #pragma GCC pop_options
>
> #pragma GCC push_options
> @@ -225,6 +233,20 @@
> #endif
> #pragma GCC pop_options
>
> +#pragma GCC push_options
> +#pragma GCC target "arch=armv8-a+ssve-aes"
> +#ifndef __ARM_FEATURE_SSVE_AES
> +#error "__ARM_FEATURE_SSVE_AES is not defined but should be!"
> +#endif
> +#pragma GCC pop_options
> +
> +#pragma GCC push_options
> +#pragma GCC target "arch=armv8-a+sve-aes2"
> +#ifndef __ARM_FEATURE_SVE_AES2
> +#error "__ARM_FEATURE_SVE_AES2 is not defined but should be!"
> +#endif
> +#pragma GCC pop_options
> +
> int
> foo (int a)
> {
> diff --git a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/test_sve_acle.h b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/test_sve_acle.h
> index aeb58ead0dd..7932a554d1e 100644
> --- a/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/test_sve_acle.h
> +++ b/gcc/testsuite/gcc.target/aarch64/sve/acle/asm/test_sve_acle.h
> @@ -687,6 +687,24 @@
> __asm volatile ("" :: "w" (RES)); \
> }
>
> +#define TEST_XN_INDEXED(NAME, TTYPE, VTYPE, CODE1, CODE2) \
> + PROTO (NAME, TTYPE, (TTYPE t, VTYPE v)) \
> + { \
> + register TTYPE z0 __asm ("z0"); \
> + register TTYPE z1 __asm ("z1"); \
> + register TTYPE z2 __asm ("z2"); \
> + register TTYPE z3 __asm ("z3"); \
> + register VTYPE z4 __asm ("z4"); \
> + register VTYPE z5 __asm ("z5"); \
> + register VTYPE z6 __asm ("z6"); \
> + register VTYPE z7 __asm ("z7"); \
> + register uint64_t x0 __asm ("x0"); \
> + __asm volatile ("" : "=r" (x0)); \
> + INVOKE (CODE1, CODE2); \
> + __asm volatile ("" :: "w" (t)); \
> + return t; \
> + }
> +
> #define TEST_DUAL_XN(NAME, TTYPE1, TTYPE2, RES, CODE1, CODE2) \
> PROTO (NAME, void, ()) \
> { \
> diff --git a/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aesd_lane_u8.c b/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aesd_lane_u8.c
> new file mode 100644
> index 00000000000..9d3591a5f39
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aesd_lane_u8.c
> @@ -0,0 +1,92 @@
> +/* { dg-do assemble { target { aarch64_asm_ssve-aes_ok && aarch64_asm_sve-aes2_ok } } } */
> +/* { dg-do compile { target { ! { aarch64_asm_ssve-aes_ok && aarch64_asm_sve-aes2_ok } } } } */
> +/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
> +
> +#include "test_sve_acle.h"
> +
> +#pragma GCC target "+sve-aes2+ssve-aes"
> +
> +/*
> +** test_aesd_lane_u8_x2:
> +** aesd {z0.b - z1.b}, {z0.b - z1.b}, z2.q\[0\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aesd_lane_u8_x2, svuint8x2_t, svuint8_t,
> + t = svaesd_lane_u8_x2 (t, v, 0),
> + t = svaesd_lane (t, v, 0))
> +
> +/*
> +** test_aesd_lane_u8_x2_lane1:
> +** aesd {z0.b - z1.b}, {z0.b - z1.b}, z2.q\[3\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aesd_lane_u8_x2_lane1, svuint8x2_t, svuint8_t,
> + t = svaesd_lane_u8_x2 (t, v, 3),
> + t = svaesd_lane (t, v, 3))
> +
> +/*
> +** test_aesd_lane_u8_x4:
> +** aesd {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[0\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aesd_lane_u8_x4, svuint8x4_t, svuint8_t,
> + t = svaesd_lane_u8_x4 (t, v, 0),
> + t = svaesd_lane (t, v, 0))
> +
> +/*
> +** test_aesd_lane_u8_x4_lane3:
> +** aesd {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[3\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aesd_lane_u8_x4_lane3, svuint8x4_t, svuint8_t,
> + t = svaesd_lane_u8_x4 (t, v, 3),
> + t = svaesd_lane (t, v, 3))
> +
> +
> +/*
> +** test_aesd_lane_u8_x2_regs_mov:
> +** mov z0.d, z3.d
> +** mov z1.d, z4.d
> +** aesd {z0.b - z1.b}, {z0.b - z1.b}, z4.q\[0\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aesd_lane_u8_x2_regs_mov, svuint8x2_t, svuint8_t,
> + t = svaesd_lane_u8_x2 (z3, z4, 0),
> + t = svaesd_lane (z3, z4, 0))
> +
> +/*
> +** test_aesd_lane_u8_x2_regs_mov_lane1:
> +** mov z0.d, z3.d
> +** mov z1.d, z4.d
> +** aesd {z0.b - z1.b}, {z0.b - z1.b}, z4.q\[3\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aesd_lane_u8_x2_regs_mov_lane1, svuint8x2_t, svuint8_t,
> + t = svaesd_lane_u8_x2 (z3, z4, 3),
> + t = svaesd_lane (z3, z4, 3))
> +
> +/*
> +** test_aesd_lane_u8_x4_regs_mov:
> +** mov z0.d, z3.d
> +** mov z1.d, z4.d
> +** mov z2.d, z5.d
> +** mov z3.d, z6.d
> +** aesd {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[0\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aesd_lane_u8_x4_regs_mov, svuint8x4_t, svuint8_t,
> + t = svaesd_lane_u8_x4 (z3, z4, 0),
> + t = svaesd_lane (z3, z4, 0))
> +
> +/*
> +** test_aesd_lane_u8_x4_regs_mov_lane3:
> +** mov z0.d, z3.d
> +** mov z1.d, z4.d
> +** mov z2.d, z5.d
> +** mov z3.d, z6.d
> +** aesd {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[3\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aesd_lane_u8_x4_regs_mov_lane3, svuint8x4_t, svuint8_t,
> + t = svaesd_lane_u8_x4 (z3, z4, 3),
> + t = svaesd_lane (z3, z4, 3))
> diff --git a/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aesdimc_lane_u8.c b/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aesdimc_lane_u8.c
> new file mode 100644
> index 00000000000..582cb3f742d
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aesdimc_lane_u8.c
> @@ -0,0 +1,91 @@
> +/* { dg-do assemble { target { aarch64_asm_ssve-aes_ok && aarch64_asm_sve-aes2_ok } } } */
> +/* { dg-do compile { target { ! { aarch64_asm_ssve-aes_ok && aarch64_asm_sve-aes2_ok } } } } */
> +/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
> +
> +#include "test_sve_acle.h"
> +
> +#pragma GCC target "+sve-aes2+ssve-aes"
> +
> +/*
> +** test_aesdimc_lane_u8_x2:
> +** aesdimc {z0.b - z1.b}, {z0.b - z1.b}, z2.q\[0\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aesdimc_lane_u8_x2, svuint8x2_t, svuint8_t,
> + t = svaesdimc_lane_u8_x2 (t, v, 0),
> + t = svaesdimc_lane (t, v, 0))
> +
> +/*
> +** test_aesdimc_lane_u8_x2_lane1:
> +** aesdimc {z0.b - z1.b}, {z0.b - z1.b}, z2.q\[3\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aesdimc_lane_u8_x2_lane1, svuint8x2_t, svuint8_t,
> + t = svaesdimc_lane_u8_x2 (t, v, 3),
> + t = svaesdimc_lane (t, v, 3))
> +
> +/*
> +** test_aesdimc_lane_u8_x4:
> +** aesdimc {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[0\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aesdimc_lane_u8_x4, svuint8x4_t, svuint8_t,
> + t = svaesdimc_lane_u8_x4 (t, v, 0),
> + t = svaesdimc_lane (t, v, 0))
> +
> +/*
> +** test_aesdimc_lane_u8_x4_lane3:
> +** aesdimc {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[3\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aesdimc_lane_u8_x4_lane3, svuint8x4_t, svuint8_t,
> + t = svaesdimc_lane_u8_x4 (t, v, 3),
> + t = svaesdimc_lane (t, v, 3))
> +
> +/*
> +** test_aesdimc_lane_u8_x2_regs_mov:
> +** mov z0.d, z3.d
> +** mov z1.d, z4.d
> +** aesdimc {z0.b - z1.b}, {z0.b - z1.b}, z4.q\[0\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aesdimc_lane_u8_x2_regs_mov, svuint8x2_t, svuint8_t,
> + t = svaesdimc_lane_u8_x2 (z3, z4, 0),
> + t = svaesdimc_lane (z3, z4, 0))
> +
> +/*
> +** test_aesdimc_lane_u8_x2_regs_mov_lane1:
> +** mov z0.d, z3.d
> +** mov z1.d, z4.d
> +** aesdimc {z0.b - z1.b}, {z0.b - z1.b}, z4.q\[3\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aesdimc_lane_u8_x2_regs_mov_lane1, svuint8x2_t, svuint8_t,
> + t = svaesdimc_lane_u8_x2 (z3, z4, 3),
> + t = svaesdimc_lane (z3, z4, 3))
> +
> +/*
> +** test_aesdimc_lane_u8_x4_regs_mov:
> +** mov z0.d, z3.d
> +** mov z1.d, z4.d
> +** mov z2.d, z5.d
> +** mov z3.d, z6.d
> +** aesdimc {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[0\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aesdimc_lane_u8_x4_regs_mov, svuint8x4_t, svuint8_t,
> + t = svaesdimc_lane_u8_x4 (z3, z4, 0),
> + t = svaesdimc_lane (z3, z4, 0))
> +
> +/*
> +** test_aesdimc_lane_u8_x4_regs_mov_lane3:
> +** mov z0.d, z3.d
> +** mov z1.d, z4.d
> +** mov z2.d, z5.d
> +** mov z3.d, z6.d
> +** aesdimc {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[3\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aesdimc_lane_u8_x4_regs_mov_lane3, svuint8x4_t, svuint8_t,
> + t = svaesdimc_lane_u8_x4 (z3, z4, 3),
> + t = svaesdimc_lane (z3, z4, 3))
> diff --git a/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aese_lane_u8.c b/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aese_lane_u8.c
> new file mode 100644
> index 00000000000..a494e072cdb
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aese_lane_u8.c
> @@ -0,0 +1,91 @@
> +/* { dg-do assemble { target { aarch64_asm_ssve-aes_ok && aarch64_asm_sve-aes2_ok } } } */
> +/* { dg-do compile { target { ! { aarch64_asm_ssve-aes_ok && aarch64_asm_sve-aes2_ok } } } } */
> +/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
> +
> +#include "test_sve_acle.h"
> +
> +#pragma GCC target "+sve-aes2+ssve-aes"
> +
> +/*
> +** test_aese_lane_u8_x2:
> +** aese {z0.b - z1.b}, {z0.b - z1.b}, z2.q\[0\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aese_lane_u8_x2, svuint8x2_t, svuint8_t,
> + t = svaese_lane_u8_x2 (t, v, 0),
> + t = svaese_lane (t, v, 0))
> +
> +/*
> +** test_aese_lane_u8_x2_lane1:
> +** aese {z0.b - z1.b}, {z0.b - z1.b}, z2.q\[3\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aese_lane_u8_x2_lane1, svuint8x2_t, svuint8_t,
> + t = svaese_lane_u8_x2 (t, v, 3),
> + t = svaese_lane (t, v, 3))
> +
> +/*
> +** test_aese_lane_u8_x4:
> +** aese {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[0\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aese_lane_u8_x4, svuint8x4_t, svuint8_t,
> + t = svaese_lane_u8_x4 (t, v, 0),
> + t = svaese_lane (t, v, 0))
> +
> +/*
> +** test_aese_lane_u8_x4_lane3:
> +** aese {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[3\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aese_lane_u8_x4_lane3, svuint8x4_t, svuint8_t,
> + t = svaese_lane_u8_x4 (t, v, 3),
> + t = svaese_lane (t, v, 3))
> +
> +/*
> +** test_aese_lane_u8_x2_regs_mov:
> +** mov z0.d, z3.d
> +** mov z1.d, z4.d
> +** aese {z0.b - z1.b}, {z0.b - z1.b}, z4.q\[0\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aese_lane_u8_x2_regs_mov, svuint8x2_t, svuint8_t,
> + t = svaese_lane_u8_x2 (z3, z4, 0),
> + t = svaese_lane (z3, z4, 0))
> +
> +/*
> +** test_aese_lane_u8_x2_regs_mov_lane1:
> +** mov z0.d, z3.d
> +** mov z1.d, z4.d
> +** aese {z0.b - z1.b}, {z0.b - z1.b}, z4.q\[3\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aese_lane_u8_x2_regs_mov_lane1, svuint8x2_t, svuint8_t,
> + t = svaese_lane_u8_x2 (z3, z4, 3),
> + t = svaese_lane (z3, z4, 3))
> +
> +/*
> +** test_aese_lane_u8_x4_regs_mov:
> +** mov z0.d, z3.d
> +** mov z1.d, z4.d
> +** mov z2.d, z5.d
> +** mov z3.d, z6.d
> +** aese {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[0\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aese_lane_u8_x4_regs_mov, svuint8x4_t, svuint8_t,
> + t = svaese_lane_u8_x4 (z3, z4, 0),
> + t = svaese_lane (z3, z4, 0))
> +
> +/*
> +** test_aese_lane_u8_x4_regs_mov_lane3:
> +** mov z0.d, z3.d
> +** mov z1.d, z4.d
> +** mov z2.d, z5.d
> +** mov z3.d, z6.d
> +** aese {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[3\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aese_lane_u8_x4_regs_mov_lane3, svuint8x4_t, svuint8_t,
> + t = svaese_lane_u8_x4 (z3, z4, 3),
> + t = svaese_lane (z3, z4, 3))
> diff --git a/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aesemc_lane_u8.c b/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aesemc_lane_u8.c
> new file mode 100644
> index 00000000000..e3cb5c29755
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/aesemc_lane_u8.c
> @@ -0,0 +1,91 @@
> +/* { dg-do assemble { target { aarch64_asm_ssve-aes_ok && aarch64_asm_sve-aes2_ok } } } */
> +/* { dg-do compile { target { ! { aarch64_asm_ssve-aes_ok && aarch64_asm_sve-aes2_ok } } } } */
> +/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
> +
> +#include "test_sve_acle.h"
> +
> +#pragma GCC target "+sve-aes2+ssve-aes"
> +
> +/*
> +** test_aesemc_lane_u8_x2:
> +** aesemc {z0.b - z1.b}, {z0.b - z1.b}, z2.q\[0\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aesemc_lane_u8_x2, svuint8x2_t, svuint8_t,
> + t = svaesemc_lane_u8_x2 (t, v, 0),
> + t = svaesemc_lane (t, v, 0))
> +
> +/*
> +** test_aesemc_lane_u8_x2_lane1:
> +** aesemc {z0.b - z1.b}, {z0.b - z1.b}, z2.q\[3\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aesemc_lane_u8_x2_lane1, svuint8x2_t, svuint8_t,
> + t = svaesemc_lane_u8_x2 (t, v, 3),
> + t = svaesemc_lane (t, v, 3))
> +
> +/*
> +** test_aesemc_lane_u8_x4:
> +** aesemc {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[0\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aesemc_lane_u8_x4, svuint8x4_t, svuint8_t,
> + t = svaesemc_lane_u8_x4 (t, v, 0),
> + t = svaesemc_lane (t, v, 0))
> +
> +/*
> +** test_aesemc_lane_u8_x4_lane3:
> +** aesemc {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[3\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aesemc_lane_u8_x4_lane3, svuint8x4_t, svuint8_t,
> + t = svaesemc_lane_u8_x4 (t, v, 3),
> + t = svaesemc_lane (t, v, 3))
> +
> +/*
> +** test_aesemc_lane_u8_x2_regs_mov:
> +** mov z0.d, z3.d
> +** mov z1.d, z4.d
> +** aesemc {z0.b - z1.b}, {z0.b - z1.b}, z4.q\[0\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aesemc_lane_u8_x2_regs_mov, svuint8x2_t, svuint8_t,
> + t = svaesemc_lane_u8_x2 (z3, z4, 0),
> + t = svaesemc_lane (z3, z4, 0))
> +
> +/*
> +** test_aesemc_lane_u8_x2_regs_mov_lane1:
> +** mov z0.d, z3.d
> +** mov z1.d, z4.d
> +** aesemc {z0.b - z1.b}, {z0.b - z1.b}, z4.q\[3\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aesemc_lane_u8_x2_regs_mov_lane1, svuint8x2_t, svuint8_t,
> + t = svaesemc_lane_u8_x2 (z3, z4, 3),
> + t = svaesemc_lane (z3, z4, 3))
> +
> +/*
> +** test_aesemc_lane_u8_x4_regs_mov:
> +** mov z0.d, z3.d
> +** mov z1.d, z4.d
> +** mov z2.d, z5.d
> +** mov z3.d, z6.d
> +** aesemc {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[0\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aesemc_lane_u8_x4_regs_mov, svuint8x4_t, svuint8_t,
> + t = svaesemc_lane_u8_x4 (z3, z4, 0),
> + t = svaesemc_lane (z3, z4, 0))
> +
> +/*
> +** test_aesemc_lane_u8_x4_regs_mov_lane3:
> +** mov z0.d, z3.d
> +** mov z1.d, z4.d
> +** mov z2.d, z5.d
> +** mov z3.d, z6.d
> +** aesemc {z0.b - z3.b}, {z0.b - z3.b}, z4.q\[3\]
> +** ret
> +*/
> +TEST_XN_INDEXED (test_aesemc_lane_u8_x4_regs_mov_lane3, svuint8x4_t, svuint8_t,
> + t = svaesemc_lane_u8_x4 (z3, z4, 3),
> + t = svaesemc_lane (z3, z4, 3))
> diff --git a/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/pmlal_pair_u64.c b/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/pmlal_pair_u64.c
> new file mode 100644
> index 00000000000..222832adff3
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/pmlal_pair_u64.c
> @@ -0,0 +1,113 @@
> +/* { dg-do assemble { target { aarch64_asm_ssve-aes_ok && aarch64_asm_sve-aes2_ok } } } */
> +/* { dg-do compile { target { ! { aarch64_asm_ssve-aes_ok && aarch64_asm_sve-aes2_ok } } } } */
> +/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
> +
> +#include "test_sve_acle.h"
> +
> +#pragma GCC target "+sve-aes2+ssve-aes"
> +
> +/*
> +**test_pmlal_pair_u64:
> +** pmlal {z0.q - z1.q}, z2.d, z2.d
> +** ret
> +*/
> +TEST_XN_INDEXED(test_pmlal_pair_u64, svuint64x2_t, svuint64_t,
> + t = svpmlal_pair_u64_x2(t, v, v),
> + t = svpmlal_pair(t, v, v))
> +
> +/*
> +**test_pmlal_pair_n_u64_regs:
> +** pmlal {z0.q - z1.q}, z4.d, z5.d
> +** ret
> +*/
> +TEST_XN_INDEXED(test_pmlal_pair_n_u64_regs, svuint64x2_t, svuint64_t,
> + t = svpmlal_pair_u64_x2(z0, z4, z5),
> + t = svpmlal_pair(z0, z4, z5))
> +
> +/*
> +**test_pmlal_pair_n_u64_regs_imm:
> +** movi d[0-9]{1,2}, #0
> +** pmlal {z0.q - z1.q}, z4.d, z[0-9]{1,2}\.d
> +** ret
> +*/
> +TEST_XN_INDEXED(test_pmlal_pair_n_u64_regs_imm, svuint64x2_t, svuint64_t,
> + t = svpmlal_pair_n_u64_x2(z0, z4, 0x0),
> + t = svpmlal_pair(z0, z4, 0x0))
> +
> +/*
> +**test_pmlal_pair_n_u64_regs_x0:
> +** mov z([0-9]{1,2})\.d, x0
> +** pmlal {z0.q - z1.q}, z4.d, z\1.d
> +** ret
> +*/
> +TEST_XN_INDEXED(test_pmlal_pair_n_u64_regs_x0, svuint64x2_t, svuint64_t,
> + t = svpmlal_pair_n_u64_x2(z0, z4, x0),
> + t = svpmlal_pair(z0, z4, x0))
> +
> +/*
> +**test_pmlal_pair_n_u64_regs_imm_1:
> +** mov z[0-9]{1,2}\.d, #65535
> +** pmlal {z0.q - z1.q}, z4.d, z[0-9]{1,2}\.d
> +** ret
> +*/
> +TEST_XN_INDEXED(test_pmlal_pair_n_u64_regs_imm_1, svuint64x2_t, svuint64_t,
> + t = svpmlal_pair_n_u64_x2(z0, z4, 0xFFFF),
> + t = svpmlal_pair(z0, z4, 0xFFFF))
> +
> +/*
> +**test_pmlal_pair_n_u64_regs_x0_1:
> +** mov z([0-9]{1,2})\.d, x0
> +** pmlal {z0.q - z1.q}, z4.d, z\1.d
> +** ret
> +*/
> +TEST_XN_INDEXED(test_pmlal_pair_n_u64_regs_x0_1, svuint64x2_t, svuint64_t,
> + t = svpmlal_pair_n_u64_x2(z0, z4, x0),
> + t = svpmlal_pair(z0, z4, x0))
> +
> +/*
> +**test_pmlal_pair_n_u64_regs_mov:
> +** movi d30, #0
> +** mov z0.d, z3.d
> +** mov z1.d, z4.d
> +** pmlal {z0.q - z1.q}, z4.d, z30.d
> +** ret
> +*/
> +TEST_XN_INDEXED(test_pmlal_pair_n_u64_regs_mov, svuint64x2_t, svuint64_t,
> + t = svpmlal_pair_n_u64_x2(z3, z4, 0x0),
> + t = svpmlal_pair(z3, z4, 0x0))
> +
> +/*
> +**test_pmlal_pair_n_u64_regs_mov_x0:
> +** mov z30.d, x0
> +** mov z0.d, z3.d
> +** mov z1.d, z4.d
> +** pmlal {z0.q - z1.q}, z4.d, z30.d
> +** ret
> +*/
> +TEST_XN_INDEXED(test_pmlal_pair_n_u64_regs_mov_x0, svuint64x2_t, svuint64_t,
> + t = svpmlal_pair_n_u64_x2(z3, z4, x0),
> + t = svpmlal_pair(z3, z4, x0))
> +
> +/*
> +**test_pmlal_pair_n_u64_regs_mov_1:
> +** mov z30.d, #65535
> +** mov z0.d, z3.d
> +** mov z1.d, z4.d
> +** pmlal {z0.q - z1.q}, z4.d, z30.d
> +** ret
> +*/
> +TEST_XN_INDEXED(test_pmlal_pair_n_u64_regs_mov_1, svuint64x2_t, svuint64_t,
> + t = svpmlal_pair_n_u64_x2(z3, z4, 0xFFFF),
> + t = svpmlal_pair(z3, z4, 0xFFFF))
> +
> +/*
> +**test_pmlal_pair_n_u64_regs_mov_x0_1:
> +** mov z30.d, x0
> +** mov z0.d, z3.d
> +** mov z1.d, z4.d
> +** pmlal {z0.q - z1.q}, z4.d, z30.d
> +** ret
> +*/
> +TEST_XN_INDEXED(test_pmlal_pair_n_u64_regs_mov_x0_1, svuint64x2_t, svuint64_t,
> + t = svpmlal_pair_n_u64_x2(z3, z4, x0),
> + t = svpmlal_pair(z3, z4, x0))
> diff --git a/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/pmull_pair_u64.c b/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/pmull_pair_u64.c
> new file mode 100644
> index 00000000000..93420ac16c1
> --- /dev/null
> +++ b/gcc/testsuite/gcc.target/aarch64/sve2/acle/asm/pmull_pair_u64.c
> @@ -0,0 +1,65 @@
> +/* { dg-do assemble { target { aarch64_asm_ssve-aes_ok && aarch64_asm_sve-aes2_ok } } } */
> +/* { dg-do compile { target { ! { aarch64_asm_ssve-aes_ok && aarch64_asm_sve-aes2_ok } } } } */
> +/* { dg-final { check-function-bodies "**" "" "-DCHECK_ASM" } } */
> +
> +#include "test_sve_acle.h"
> +
> +#pragma GCC target "+sve-aes2+ssve-aes"
> +
> +/*
> +**test_pmull_pair_u64:
> +** pmull {z0.q - z1.q}, z2.d, z2.d
> +** ret
> +*/
> +TEST_XN_INDEXED(test_pmull_pair_u64, svuint64x2_t, svuint64_t,
> + t = svpmull_pair_u64_x2(v, v),
> + t = svpmull_pair(v, v))
> +
> +/*
> +**test_pmull_pair_n_u64:
> +** movi d([0-9]{1,2}), #0
> +** pmull {z0.q - z1.q}, z2.d, z\1.d
> +** ret
> +*/
> +TEST_XN_INDEXED(test_pmull_pair_n_u64, svuint64x2_t, svuint64_t,
> + t = svpmull_pair_n_u64_x2(v, 0x0),
> + t = svpmull_pair(v, 0x0))
> +
> +/*
> +**test_pmull_pair_n_u64_x0:
> +** mov z([0-9]{1,2})\.d, x0
> +** pmull {z0.q - z1.q}, z2.d, z\1.d
> +** ret
> +*/
> +TEST_XN_INDEXED(test_pmull_pair_n_u64_x0, svuint64x2_t, svuint64_t,
> + t = svpmull_pair_n_u64_x2(v, x0),
> + t = svpmull_pair(v, x0))
> +
> +/*
> +**test_pmull_pair_u64_regs:
> +** pmull {z0.q - z1.q}, z4.d, z5.d
> +** ret
> +*/
> +TEST_XN_INDEXED(test_pmull_pair_u64_regs, svuint64x2_t, svuint64_t,
> + t = svpmull_pair_u64_x2(z4, z5),
> + t = svpmull_pair(z4, z5))
> +
> +/*
> +**test_pmull_pair_n_u64_regs:
> +** movi d([0-9]{1,2}), #0
> +** pmull {z0.q - z1.q}, z4.d, z\1.d
> +** ret
> +*/
> +TEST_XN_INDEXED(test_pmull_pair_n_u64_regs, svuint64x2_t, svuint64_t,
> + t = svpmull_pair_n_u64_x2(z4, 0x0),
> + t = svpmull_pair(z4, 0x0))
> +
> +/*
> +**test_pmull_pair_n_u64_regs_x0:
> +** mov z([0-9]{1,2})\.d, x0
> +** pmull {z0.q - z1.q}, z4.d, z\1.d
> +** ret
> +*/
> +TEST_XN_INDEXED(test_pmull_pair_n_u64_regs_x0, svuint64x2_t, svuint64_t,
> + t = svpmull_pair_n_u64_x2(z4, x0),
> + t = svpmull_pair(z4, x0))
> diff --git a/gcc/testsuite/lib/target-supports.exp b/gcc/testsuite/lib/target-supports.exp
> index 42cbb5ce3df..de43c767801 100644
> --- a/gcc/testsuite/lib/target-supports.exp
> +++ b/gcc/testsuite/lib/target-supports.exp
> @@ -13045,8 +13045,8 @@ proc check_effective_target_aarch64_gas_has_dtprel_reloc { } {
> set exts {
> "bf16" "cmpbr" "crc" "crypto" "dotprod" "f32mm" "f64mm" "faminmax"
> "fp" "fp8" "fp8dot2" "fp8dot4" "fp8fma" "i8mm" "ls64" "lse" "lut"
> - "sb" "simd" "sve-b16b16" "sve" "sve2" "sve-sm4" "sve-aes" "sve-bitperm"
> - "sve-sha3" "f8f16mm" "f8f32mm" "sve-f16f32mm"
> + "sb" "simd" "sve-b16b16" "sve" "sve2" "sve-sm4" "sve-aes" "sve-aes2"
> + "sve-bitperm" "sve-sha3" "f8f16mm" "f8f32mm" "sve-f16f32mm"
> "sme-f8f16" "sme-f8f32"
> "sme-b16b16" "sme-f16f16" "sme-i16i64" "sme" "sme2" "sme2p1" "sme2p2"
> "sme2p3"
> --
> 2.43.0
>