[PATCH v4 0/2] arm: recognize vdupq idioms [PR124043]
Richard Earnshaw via Sourceware Forge <[email protected]> Mon, 03 Aug 2026 13:59:41 +0000
| Newsgroups | gmane.comp.gcc.patches |
|---|---|
| Message-ID | <bmm.hl01du4maw.gcc.gcc.rearnsha.208.4.0@forge-stage.sourceware.org> |
Hi gcc-patches mailing list, Richard Earnshaw via Sourceware Forge <[email protected]> has requested that the following forgejo pull request be published on the mailing list. Created on: 2026-07-30 15:40:01+00:00 Latest update: 2026-08-03 13:59:42+00:00 Changes: 8 changed files, 314 additions, 74 deletions Head revision: rearnsha/gcc ref vdupq commit 8fa942b7a17deb4f99f1a4e7b5fcff0e991c133d Base revision: gcc/gcc ref trunk commit 3db020600d47180dc097f405d1dd7cfcafe24d43 r17-2898-g3db020600d4718 Merge base: 3db020600d47180dc097f405d1dd7cfcafe24d43 Full diff url: https://forge.sourceware.org/gcc/gcc/pulls/208.diff Discussion: https://forge.sourceware.org/gcc/gcc/pulls/208 Requested Reviewers: azoff Changes since v1: - address comments from Christophe and Torbjorn - Handle core regs to HF and BF vector modes - Disambiguate the pattern names Changes since v2: - Fix some additional tests that were failing with -mfloat-abi=softfp Changes since v3: - Move the fixes from v2 to the right patch in the sequence. Changed files: - M: gcc/config/arm/neon.md - M: gcc/config/arm/vfp.md - M: gcc/testsuite/gcc.target/arm/armv8_2-fp16-neon-1.c - M: gcc/testsuite/gcc.target/arm/armv8_2-fp16-neon-2.c - M: gcc/testsuite/gcc.target/arm/crypto-vsha1cq_u32.c - M: gcc/testsuite/gcc.target/arm/crypto-vsha1h_u32.c - M: gcc/testsuite/gcc.target/arm/crypto-vsha1mq_u32.c - M: gcc/testsuite/gcc.target/arm/crypto-vsha1pq_u32.c Richard Earnshaw (2): arm: handle neon vec_dup from select of 128-bit vector arm: Thumb2 reg preferencing for vfp variant of movsi [PR124043] gcc/config/arm/neon.md | 164 ++++++++++++++---- gcc/config/arm/vfp.md | 8 +- .../gcc.target/arm/armv8_2-fp16-neon-1.c | 114 +++++++++--- .../gcc.target/arm/armv8_2-fp16-neon-2.c | 90 +++++++++- .../gcc.target/arm/crypto-vsha1cq_u32.c | 3 +- .../gcc.target/arm/crypto-vsha1h_u32.c | 3 +- .../gcc.target/arm/crypto-vsha1mq_u32.c | 3 +- .../gcc.target/arm/crypto-vsha1pq_u32.c | 3 +- 8 files changed, 314 insertions(+), 74 deletions(-) Range-diff against v3: 2: 5f1f4582e0363 ! 1: c6464a1d252a7 arm: Thumb2 reg preferencing for vfp variant of movsi [PR124043] @@ Metadata Author: Richard Earnshaw <[email protected]> ## Commit message ## - arm: Thumb2 reg preferencing for vfp variant of movsi [PR124043] + arm: handle neon vec_dup from select of 128-bit vector - This pattern currently masks a lot of alternatives from register - preferencing, but this can cause the register allocator to prefer to - spill and reload from the stack when transferring values between core - and VFP registers. (This is probably exacerbated by LRA also chosing to - disparage such alternatives when doing final register selection.) This - patch is not a complete rework of the preferences, but does try to bring - the alternatives closer to the way we handle registers when compiling in - arm mode (A32). + The Neon instruction set lacks a direct vdup from a lane in a 128-bit + vector; but one isn't needed because the lane is a constant and we can + handle the appropriate half of a 128-bit register simply by selecting + that during output. - This pattern needs reworking - it probably hasn't been looked at - properly since we moved to LRA. + Additionally, we can also handle core registers as the source operand + for both 64-bit and 128-bit sources without needing to copy the entire + vector to VFP/SIMD regs; at most a simple shift is needed to extract + the appropriate lane to the lower bits of a scratch core reg, but in + some cases we can use the core reg directly (when no shift is needed) + since the upper bits are ignored. + + I've also disambiguated the two paterns named + neon_vdup_lane<mode>_internal by inserting the iterator name into the + pattern. This doesn't change anything in terms of generated code, but + makes the pattern names in the MD file unique. gcc/ChangeLog: - PR target/124043 - * config/arm/vfp.md (thumb2_movsi_vfp): Don't hide the t->r - alternative from register preferencing. + * config/arm/neon.md (neon_vdup_lane<VDQW:mode>_internal): + Handle core registers as the input vector operand by splitting. + (neon_vdup_lane<VHFBF:mode>_internal): Likewise. + (neon_vdupq_lane<VQ2BF:mode>_internal): New pattern. gcc/testsuite/ChangeLog: - PR target/124043 - * gcc.target/arm/crypto-vsha1cq_u32.c: Remove xfail on vdup. + * gcc.target/arm/crypto-vsha1cq_u32.c: Don't expect a vmov.32 in + the generated code. * gcc.target/arm/crypto-vsha1h_u32.c: Likewise. * gcc.target/arm/crypto-vsha1mq_u32.c: Likewise. * gcc.target/arm/crypto-vsha1pq_u32.c: Likewise. - * gcc.target/arm/armv8_2-fp16-neon-1.c: Adjust expected output. + * gcc.target/arm/armv8_2-fp16-neon-1.c: Update expected output. * gcc.target/arm/armv8_2-fp16-neon-2.c: Likewise. - ## gcc/config/arm/vfp.md ## -@@ - ;; is chosen with length 2 when the instruction is predicated for - ;; arm_restrict_it. - (define_insn "*thumb2_movsi_vfp" -- [(set (match_operand:SI 0 "nonimmediate_operand" "=rk,r,l,r,r,l,*hk,m,*m,*t,\ -- r,*t,*t,*Uv, Up, r,Uf,r") -- (match_operand:SI 1 "general_operand" "rk,I,Py,K,j,mi,*mi,l,*hk,r,*t,\ -- *t,*UvTu,*t, r, Up,r,Uf"))] -+ [(set (match_operand:SI 0 "nonimmediate_operand" -+ "=rk,r,l, r,r,l, *hk,m,*m, *t,r,*t,*t, *Uv,Up,r, Uf,r") -+ (match_operand:SI 1 "general_operand" -+ "rk, I,Py,K,j,mi,*mi,l,*hk,r, t,*t,*UvTu,*t, r, Up,r, Uf"))] - "TARGET_THUMB2 && TARGET_VFP_BASE - && ( s_register_operand (operands[0], SImode) - || s_register_operand (operands[1], SImode))" + ## gcc/config/arm/neon.md ## +@@ gcc/config/arm/neon.md: if (BYTES_BIG_ENDIAN) + (set_attr "type" "multiple")] + ) + +-(define_insn "neon_vdup_lane<mode>_internal" +- [(set (match_operand:VDQW 0 "s_register_operand" "=w") +- (vec_duplicate:VDQW +- (vec_select:<V_elem> +- (match_operand:<V_double_vector_mode> 1 "s_register_operand" "w") +- (parallel [(match_operand:SI 2 "immediate_operand" "i")]))))] ++(define_insn_and_split "neon_vdup_lane<VDQW:mode>_internal" ++ [(set (match_operand:VDQW 0 "s_register_operand" "=w,w") ++ (vec_duplicate:VDQW ++ (vec_select:<V_elem> ++ (match_operand:<V_double_vector_mode> 1 "s_register_operand" "w,r") ++ (parallel [(match_operand:SI 2 "immediate_operand" "i,i")])))) ++ (clobber (match_scratch:<V_elem> 3 "=X,r"))] + "TARGET_NEON" +-{ +- if (BYTES_BIG_ENDIAN) +- { +- int elt = INTVAL (operands[2]); ++ { ++ if (REGNO (operands[1]) <= LAST_ARM_REGNUM) ++ return "#"; ++ if (BYTES_BIG_ENDIAN) ++ { ++ int elt = INTVAL (operands[2]); ++ elt = GET_MODE_NUNITS (<V_double_vector_mode>mode) - 1 - elt; ++ operands[2] = GEN_INT (elt); ++ } ++ if (<Is_d_reg>) ++ return "vdup.<V_sz_elem>\t%P0, %P1[%c2]"; ++ else ++ return "vdup.<V_sz_elem>\t%q0, %P1[%c2]"; ++ } ++ "&& REGNO (operands[1]) <= LAST_ARM_REGNUM" ++ [(set (match_dup 0) ++ (vec_duplicate:VDQW (match_dup 3)))] ++ { ++ unsigned HOST_WIDE_INT elt = UINTVAL (operands[2]); ++ if (BYTES_BIG_ENDIAN) + elt = GET_MODE_NUNITS (<V_double_vector_mode>mode) - 1 - elt; +- operands[2] = GEN_INT (elt); +- } +- if (<Is_d_reg>) +- return "vdup.<V_sz_elem>\t%P0, %P1[%c2]"; +- else +- return "vdup.<V_sz_elem>\t%q0, %P1[%c2]"; +-} +- [(set_attr "type" "neon_dup<q>")] ++ unsigned HOST_WIDE_INT size = GET_MODE_SIZE (<V_elem>mode); ++ int base_regno = REGNO (operands[1]); ++ int regno = (base_regno ++ + subreg_regno_offset (base_regno, <V_double_vector_mode>mode, ++ elt * size, SImode)); ++ unsigned HOST_WIDE_INT offset = (elt * size) % GET_MODE_SIZE (SImode); ++ if (offset != 0) ++ { ++ gcc_assert (offset < 4); ++ rtx reg = gen_rtx_REG (SImode, regno); ++ rtx shift = gen_rtx_LSHIFTRT (SImode, reg, ++ GEN_INT (offset * BITS_PER_UNIT)); ++ emit_move_insn (gen_rtx_SUBREG (SImode, operands[3], 0), ++ shift); ++ } ++ else ++ operands[3] = gen_rtx_REG (<V_elem>mode, regno); ++ } ++ [(set_attr "length" "4,8") ++ (set_attr "type" "neon_dup<q>")] + ) + +-(define_insn "neon_vdup_lane<mode>_internal" +- [(set (match_operand:VHFBF 0 "s_register_operand" "=w") +- (vec_duplicate:VHFBF +- (vec_select:<V_elem> +- (match_operand:<V_double_vector_mode> 1 "s_register_operand" "w") +- (parallel [(match_operand:SI 2 "immediate_operand" "i")]))))] +- "TARGET_NEON && (TARGET_FP16 || TARGET_BF16_SIMD)" +-{ +- if (BYTES_BIG_ENDIAN) +- { +- int elt = INTVAL (operands[2]); ++; There isn't an intrinsic for this, but the compiler can generate it ++; idomatically from other operations. ++(define_insn_and_split "neon_vdupq_lane<VQ2BF:mode>_internal" ++ [(set (match_operand:VQ2BF 0 "s_register_operand" "=w,w") ++ (vec_duplicate:VQ2BF ++ (vec_select:<V_elem> ++ (match_operand:VQ2BF 1 "s_register_operand" "w,r") ++ (parallel [(match_operand:SI 2 "immediate_operand" "i,i")])))) ++ (clobber (match_scratch:<V_elem> 3 "=X,r"))] ++ "TARGET_NEON" ++ "#" ++ "" ++ [(parallel ++ [(set (match_dup 0) ++ (vec_duplicate:VQ2BF ++ (vec_select:<V_elem> (match_dup 1) (parallel [(match_dup 2)])))) ++ (clobber (match_dup 3))])] ++ { ++ HOST_WIDE_INT elt = INTVAL (operands[2]); ++ if (elt >= GET_MODE_NUNITS (<MODE>mode) / 2) ++ { ++ elt -= GET_MODE_NUNITS (<MODE>mode) / 2; ++ operands[1] = simplify_gen_subreg (<V_HALF>mode, operands[1], ++ <MODE>mode, ++ GET_MODE_SIZE (<V_HALF>mode)); ++ operands[2] = GEN_INT (elt); ++ } ++ else ++ operands[1] = gen_lowpart (<V_HALF>mode, operands[1]); ++ } ++ [(set_attr "type" "neon_dup<q>") ++ (set_attr "length" "4,8")] ++) ++ ++(define_insn_and_split "neon_vdup_lane<VHFBF:mode>_internal" ++ [(set (match_operand:VHFBF 0 "s_register_operand" "=w,w") ++ (vec_duplicate:VHFBF ++ (vec_select:<V_elem> ++ (match_operand:<V_double_vector_mode> 1 "s_register_operand" "w,r") ++ (parallel [(match_operand:SI 2 "immediate_operand" "i,i")])))) ++ (clobber (match_scratch:<V_elem> 3 "=X,r"))] ++ "TARGET_NEON && (TARGET_FP16 || TARGET_BF16_SIMD)" ++ { ++ if (BYTES_BIG_ENDIAN) ++ { ++ int elt = INTVAL (operands[2]); ++ elt = GET_MODE_NUNITS (<V_double_vector_mode>mode) - 1 - elt; ++ operands[2] = GEN_INT (elt); ++ } ++ if (<Is_d_reg>) ++ return "vdup.<V_sz_elem>\t%P0, %P1[%c2]"; ++ else ++ return "vdup.<V_sz_elem>\t%q0, %P1[%c2]"; ++ } ++ "&& REGNO (operands[1]) <= LAST_ARM_REGNUM" ++ [(set (match_dup 0) ++ (vec_duplicate:VHFBF (match_dup 3)))] ++ { ++ unsigned HOST_WIDE_INT elt = UINTVAL (operands[2]); ++ if (BYTES_BIG_ENDIAN) + elt = GET_MODE_NUNITS (<V_double_vector_mode>mode) - 1 - elt; +- operands[2] = GEN_INT (elt); +- } +- if (<Is_d_reg>) +- return "vdup.<V_sz_elem>\t%P0, %P1[%c2]"; +- else +- return "vdup.<V_sz_elem>\t%q0, %P1[%c2]"; +-} +- [(set_attr "type" "neon_dup<q>")] ++ unsigned HOST_WIDE_INT size = GET_MODE_SIZE (<V_elem>mode); ++ int base_regno = REGNO (operands[1]); ++ int regno = (base_regno ++ + subreg_regno_offset (base_regno, <V_double_vector_mode>mode, ++ elt * size, SImode)); ++ unsigned HOST_WIDE_INT offset = (elt * size) % GET_MODE_SIZE (SImode); ++ if (offset != 0) ++ { ++ gcc_assert (offset < 4); ++ rtx reg = gen_rtx_REG (SImode, regno); ++ rtx shift = gen_rtx_LSHIFTRT (SImode, reg, ++ GEN_INT (offset * BITS_PER_UNIT)); ++ emit_move_insn (gen_rtx_SUBREG (SImode, operands[3], 0), ++ shift); ++ } ++ else ++ operands[3] = gen_rtx_REG (<V_elem>mode, regno); ++ } ++ [(set_attr "length" "4,8") ++ (set_attr "type" "neon_dup<q>")] + ) + + (define_expand "neon_vdup_lane<mode>" ## gcc/testsuite/gcc.target/arm/armv8_2-fp16-neon-1.c ## @@ @@ gcc/testsuite/gcc.target/arm/armv8_2-fp16-neon-2.c: test_vmov_n_f16 (float16_t a test_vext_f16 (float16x4_t a, float16x4_t b) ## gcc/testsuite/gcc.target/arm/crypto-vsha1cq_u32.c ## -@@ gcc/testsuite/gcc.target/arm/crypto-vsha1cq_u32.c: uint32_t foo (void) - TEST_SHA1C_VEC_SELECT (GET_LANE) +@@ gcc/testsuite/gcc.target/arm/crypto-vsha1cq_u32.c: TEST_SHA1C_VEC_SELECT (GET_LANE) /* { dg-final { scan-assembler-times {sha1c.32\tq[0-9]+, q[0-9]+} 5 } } */ --/* { dg-final { scan-assembler-times {vdup.32\tq[0-9]+, (?:r[0-9]+|d[0-9]+\[[0-9]+\])} 4 { xfail { arm_thumb2 && arm_hf_eabi } } } } */ -+/* { dg-final { scan-assembler-times {vdup.32\tq[0-9]+, (?:r[0-9]+|d[0-9]+\[[0-9]+\])} 4 } } */ + /* { dg-final { scan-assembler-times {vdup.32\tq[0-9]+, (?:r[0-9]+|d[0-9]+\[[0-9]+\])} 4 { xfail { arm_thumb2 && arm_hf_eabi } } } } */ +-/* { dg-final { scan-assembler-times {vmov.32\tr[0-9]+, d[0-9]+\[[0-9]+\]} 3 } } */ ## gcc/testsuite/gcc.target/arm/crypto-vsha1h_u32.c ## -@@ gcc/testsuite/gcc.target/arm/crypto-vsha1h_u32.c: uint32_t foo (void) - TEST_SHA1H_VEC_SELECT (GET_LANE) +@@ gcc/testsuite/gcc.target/arm/crypto-vsha1h_u32.c: TEST_SHA1H_VEC_SELECT (GET_LANE) /* { dg-final { scan-assembler-times {sha1h.32\tq[0-9]+, q[0-9]+} 5 } } */ --/* { dg-final { scan-assembler-times {vdup.32\tq[0-9]+, (?:r[0-9]+|d[0-9]+\[[0-9]+\])} 4 { xfail { arm_thumb2 && arm_hf_eabi } } } } */ -+/* { dg-final { scan-assembler-times {vdup.32\tq[0-9]+, (?:r[0-9]+|d[0-9]+\[[0-9]+\])} 4 } } */ + /* { dg-final { scan-assembler-times {vdup.32\tq[0-9]+, (?:r[0-9]+|d[0-9]+\[[0-9]+\])} 4 { xfail { arm_thumb2 && arm_hf_eabi } } } } */ +-/* { dg-final { scan-assembler-times {vmov.32\tr[0-9]+, d[0-9]+\[[0-9]+\]} 3 } } */ ## gcc/testsuite/gcc.target/arm/crypto-vsha1mq_u32.c ## -@@ gcc/testsuite/gcc.target/arm/crypto-vsha1mq_u32.c: uint32_t foo (void) - TEST_SHA1M_VEC_SELECT (GET_LANE) +@@ gcc/testsuite/gcc.target/arm/crypto-vsha1mq_u32.c: TEST_SHA1M_VEC_SELECT (GET_LANE) /* { dg-final { scan-assembler-times {sha1m.32\tq[0-9]+, q[0-9]+} 5 } } */ --/* { dg-final { scan-assembler-times {vdup.32\tq[0-9]+, (?:r[0-9]+|d[0-9]+\[[0-9]+\])} 4 { xfail { arm_thumb2 && arm_hf_eabi } } } } */ -+/* { dg-final { scan-assembler-times {vdup.32\tq[0-9]+, (?:r[0-9]+|d[0-9]+\[[0-9]+\])} 4 } } */ + /* { dg-final { scan-assembler-times {vdup.32\tq[0-9]+, (?:r[0-9]+|d[0-9]+\[[0-9]+\])} 4 { xfail { arm_thumb2 && arm_hf_eabi } } } } */ +-/* { dg-final { scan-assembler-times {vmov.32\tr[0-9]+, d[0-9]+\[[0-9]+\]} 3 } } */ ## gcc/testsuite/gcc.target/arm/crypto-vsha1pq_u32.c ## -@@ gcc/testsuite/gcc.target/arm/crypto-vsha1pq_u32.c: uint32_t foo (void) - TEST_SHA1P_VEC_SELECT (GET_LANE) +@@ gcc/testsuite/gcc.target/arm/crypto-vsha1pq_u32.c: TEST_SHA1P_VEC_SELECT (GET_LANE) /* { dg-final { scan-assembler-times {sha1p.32\tq[0-9]+, q[0-9]+} 5 } } */ --/* { dg-final { scan-assembler-times {vdup.32\tq[0-9]+, (?:r[0-9]+|d[0-9]+\[[0-9]+\])} 4 { xfail { arm_thumb2 && arm_hf_eabi } } } } */ -+/* { dg-final { scan-assembler-times {vdup.32\tq[0-9]+, (?:r[0-9]+|d[0-9]+\[[0-9]+\])} 4 } } */ + /* { dg-final { scan-assembler-times {vdup.32\tq[0-9]+, (?:r[0-9]+|d[0-9]+\[[0-9]+\])} 4 { xfail { arm_thumb2 && arm_hf_eabi } } } } */ +-/* { dg-final { scan-assembler-times {vmov.32\tr[0-9]+, d[0-9]+\[[0-9]+\]} 3 } } */ 1: b70a0bc7a64f5 ! 2: 8fa942b7a17de arm: handle neon vec_dup from select of 128-bit vector @@ Metadata Author: Richard Earnshaw <[email protected]> ## Commit message ## - arm: handle neon vec_dup from select of 128-bit vector + arm: Thumb2 reg preferencing for vfp variant of movsi [PR124043] - The Neon instruction set lacks a direct vdup from a lane in a 128-bit - vector; but one isn't needed because the lane is a constant and we can - handle the appropriate half of a 128-bit register simply by selecting - that during output. + This pattern currently masks a lot of alternatives from register + preferencing, but this can cause the register allocator to prefer to + spill and reload from the stack when transferring values between core + and VFP registers. (This is probably exacerbated by LRA also chosing to + disparage such alternatives when doing final register selection.) This + patch is not a complete rework of the preferences, but does try to bring + the alternatives closer to the way we handle registers when compiling in + arm mode (A32). - Additionally, we can also handle core registers as the source operand - for both 64-bit and 128-bit sources without needing to copy the entire - vector to VFP/SIMD regs; at most a simple shift is needed to extract - the appropriate lane to the lower bits of a scratch core reg, but in - some cases we can use the core reg directly (when no shift is needed) - since the upper bits are ignored. - - I've also disambiguated the two paterns named - neon_vdup_lane<mode>_internal by inserting the iterator name into the - pattern. This doesn't change anything in terms of generated code, but - makes the pattern names in the MD file unique. + This pattern needs reworking - it probably hasn't been looked at + properly since we moved to LRA. gcc/ChangeLog: - * config/arm/neon.md (neon_vdup_lane<VDQW:mode>_internal): - Handle core registers as the input vector operand by splitting. - (neon_vdup_lane<VHFBF:mode>_internal): Likewise. - (neon_vdupq_lane<VQ2BF:mode>_internal): New pattern. + PR target/124043 + * config/arm/vfp.md (thumb2_movsi_vfp): Don't hide the t->r + alternative from register preferencing. gcc/testsuite/ChangeLog: - * gcc.target/arm/crypto-vsha1cq_u32.c: Don't expect a vmov.32 in - the generated code. + PR target/124043 + * gcc.target/arm/crypto-vsha1cq_u32.c: Remove xfail on vdup. * gcc.target/arm/crypto-vsha1h_u32.c: Likewise. * gcc.target/arm/crypto-vsha1mq_u32.c: Likewise. * gcc.target/arm/crypto-vsha1pq_u32.c: Likewise. - ## gcc/config/arm/neon.md ## -@@ gcc/config/arm/neon.md: if (BYTES_BIG_ENDIAN) - (set_attr "type" "multiple")] - ) - --(define_insn "neon_vdup_lane<mode>_internal" -- [(set (match_operand:VDQW 0 "s_register_operand" "=w") -- (vec_duplicate:VDQW -- (vec_select:<V_elem> -- (match_operand:<V_double_vector_mode> 1 "s_register_operand" "w") -- (parallel [(match_operand:SI 2 "immediate_operand" "i")]))))] -+(define_insn_and_split "neon_vdup_lane<VDQW:mode>_internal" -+ [(set (match_operand:VDQW 0 "s_register_operand" "=w,w") -+ (vec_duplicate:VDQW -+ (vec_select:<V_elem> -+ (match_operand:<V_double_vector_mode> 1 "s_register_operand" "w,r") -+ (parallel [(match_operand:SI 2 "immediate_operand" "i,i")])))) -+ (clobber (match_scratch:<V_elem> 3 "=X,r"))] - "TARGET_NEON" --{ -- if (BYTES_BIG_ENDIAN) -- { -- int elt = INTVAL (operands[2]); -+ { -+ if (REGNO (operands[1]) <= LAST_ARM_REGNUM) -+ return "#"; -+ if (BYTES_BIG_ENDIAN) -+ { -+ int elt = INTVAL (operands[2]); -+ elt = GET_MODE_NUNITS (<V_double_vector_mode>mode) - 1 - elt; -+ operands[2] = GEN_INT (elt); -+ } -+ if (<Is_d_reg>) -+ return "vdup.<V_sz_elem>\t%P0, %P1[%c2]"; -+ else -+ return "vdup.<V_sz_elem>\t%q0, %P1[%c2]"; -+ } -+ "&& REGNO (operands[1]) <= LAST_ARM_REGNUM" -+ [(set (match_dup 0) -+ (vec_duplicate:VDQW (match_dup 3)))] -+ { -+ unsigned HOST_WIDE_INT elt = UINTVAL (operands[2]); -+ if (BYTES_BIG_ENDIAN) - elt = GET_MODE_NUNITS (<V_double_vector_mode>mode) - 1 - elt; -- operands[2] = GEN_INT (elt); -- } -- if (<Is_d_reg>) -- return "vdup.<V_sz_elem>\t%P0, %P1[%c2]"; -- else -- return "vdup.<V_sz_elem>\t%q0, %P1[%c2]"; --} -- [(set_attr "type" "neon_dup<q>")] -+ unsigned HOST_WIDE_INT size = GET_MODE_SIZE (<V_elem>mode); -+ int base_regno = REGNO (operands[1]); -+ int regno = (base_regno -+ + subreg_regno_offset (base_regno, <V_double_vector_mode>mode, -+ elt * size, SImode)); -+ unsigned HOST_WIDE_INT offset = (elt * size) % GET_MODE_SIZE (SImode); -+ if (offset != 0) -+ { -+ gcc_assert (offset < 4); -+ rtx reg = gen_rtx_REG (SImode, regno); -+ rtx shift = gen_rtx_LSHIFTRT (SImode, reg, -+ GEN_INT (offset * BITS_PER_UNIT)); -+ emit_move_insn (gen_rtx_SUBREG (SImode, operands[3], 0), -+ shift); -+ } -+ else -+ operands[3] = gen_rtx_REG (<V_elem>mode, regno); -+ } -+ [(set_attr "length" "4,8") -+ (set_attr "type" "neon_dup<q>")] - ) - --(define_insn "neon_vdup_lane<mode>_internal" -- [(set (match_operand:VHFBF 0 "s_register_operand" "=w") -- (vec_duplicate:VHFBF -- (vec_select:<V_elem> -- (match_operand:<V_double_vector_mode> 1 "s_register_operand" "w") -- (parallel [(match_operand:SI 2 "immediate_operand" "i")]))))] -- "TARGET_NEON && (TARGET_FP16 || TARGET_BF16_SIMD)" --{ -- if (BYTES_BIG_ENDIAN) -- { -- int elt = INTVAL (operands[2]); -+; There isn't an intrinsic for this, but the compiler can generate it -+; idomatically from other operations. -+(define_insn_and_split "neon_vdupq_lane<VQ2BF:mode>_internal" -+ [(set (match_operand:VQ2BF 0 "s_register_operand" "=w,w") -+ (vec_duplicate:VQ2BF -+ (vec_select:<V_elem> -+ (match_operand:VQ2BF 1 "s_register_operand" "w,r") -+ (parallel [(match_operand:SI 2 "immediate_operand" "i,i")])))) -+ (clobber (match_scratch:<V_elem> 3 "=X,r"))] -+ "TARGET_NEON" -+ "#" -+ "" -+ [(parallel -+ [(set (match_dup 0) -+ (vec_duplicate:VQ2BF -+ (vec_select:<V_elem> (match_dup 1) (parallel [(match_dup 2)])))) -+ (clobber (match_dup 3))])] -+ { -+ HOST_WIDE_INT elt = INTVAL (operands[2]); -+ if (elt >= GET_MODE_NUNITS (<MODE>mode) / 2) -+ { -+ elt -= GET_MODE_NUNITS (<MODE>mode) / 2; -+ operands[1] = simplify_gen_subreg (<V_HALF>mode, operands[1], -+ <MODE>mode, -+ GET_MODE_SIZE (<V_HALF>mode)); -+ operands[2] = GEN_INT (elt); -+ } -+ else -+ operands[1] = gen_lowpart (<V_HALF>mode, operands[1]); -+ } -+ [(set_attr "type" "neon_dup<q>") -+ (set_attr "length" "4,8")] -+) -+ -+(define_insn_and_split "neon_vdup_lane<VHFBF:mode>_internal" -+ [(set (match_operand:VHFBF 0 "s_register_operand" "=w,w") -+ (vec_duplicate:VHFBF -+ (vec_select:<V_elem> -+ (match_operand:<V_double_vector_mode> 1 "s_register_operand" "w,r") -+ (parallel [(match_operand:SI 2 "immediate_operand" "i,i")])))) -+ (clobber (match_scratch:<V_elem> 3 "=X,r"))] -+ "TARGET_NEON && (TARGET_FP16 || TARGET_BF16_SIMD)" -+ { -+ if (BYTES_BIG_ENDIAN) -+ { -+ int elt = INTVAL (operands[2]); -+ elt = GET_MODE_NUNITS (<V_double_vector_mode>mode) - 1 - elt; -+ operands[2] = GEN_INT (elt); -+ } -+ if (<Is_d_reg>) -+ return "vdup.<V_sz_elem>\t%P0, %P1[%c2]"; -+ else -+ return "vdup.<V_sz_elem>\t%q0, %P1[%c2]"; -+ } -+ "&& REGNO (operands[1]) <= LAST_ARM_REGNUM" -+ [(set (match_dup 0) -+ (vec_duplicate:VHFBF (match_dup 3)))] -+ { -+ unsigned HOST_WIDE_INT elt = UINTVAL (operands[2]); -+ if (BYTES_BIG_ENDIAN) - elt = GET_MODE_NUNITS (<V_double_vector_mode>mode) - 1 - elt; -- operands[2] = GEN_INT (elt); -- } -- if (<Is_d_reg>) -- return "vdup.<V_sz_elem>\t%P0, %P1[%c2]"; -- else -- return "vdup.<V_sz_elem>\t%q0, %P1[%c2]"; --} -- [(set_attr "type" "neon_dup<q>")] -+ unsigned HOST_WIDE_INT size = GET_MODE_SIZE (<V_elem>mode); -+ int base_regno = REGNO (operands[1]); -+ int regno = (base_regno -+ + subreg_regno_offset (base_regno, <V_double_vector_mode>mode, -+ elt * size, SImode)); -+ unsigned HOST_WIDE_INT offset = (elt * size) % GET_MODE_SIZE (SImode); -+ if (offset != 0) -+ { -+ gcc_assert (offset < 4); -+ rtx reg = gen_rtx_REG (SImode, regno); -+ rtx shift = gen_rtx_LSHIFTRT (SImode, reg, -+ GEN_INT (offset * BITS_PER_UNIT)); -+ emit_move_insn (gen_rtx_SUBREG (SImode, operands[3], 0), -+ shift); -+ } -+ else -+ operands[3] = gen_rtx_REG (<V_elem>mode, regno); -+ } -+ [(set_attr "length" "4,8") -+ (set_attr "type" "neon_dup<q>")] - ) - - (define_expand "neon_vdup_lane<mode>" + ## gcc/config/arm/vfp.md ## +@@ + ;; is chosen with length 2 when the instruction is predicated for + ;; arm_restrict_it. + (define_insn "*thumb2_movsi_vfp" +- [(set (match_operand:SI 0 "nonimmediate_operand" "=rk,r,l,r,r,l,*hk,m,*m,*t,\ +- r,*t,*t,*Uv, Up, r,Uf,r") +- (match_operand:SI 1 "general_operand" "rk,I,Py,K,j,mi,*mi,l,*hk,r,*t,\ +- *t,*UvTu,*t, r, Up,r,Uf"))] ++ [(set (match_operand:SI 0 "nonimmediate_operand" ++ "=rk,r,l, r,r,l, *hk,m,*m, *t,r,*t,*t, *Uv,Up,r, Uf,r") ++ (match_operand:SI 1 "general_operand" ++ "rk, I,Py,K,j,mi,*mi,l,*hk,r, t,*t,*UvTu,*t, r, Up,r, Uf"))] + "TARGET_THUMB2 && TARGET_VFP_BASE + && ( s_register_operand (operands[0], SImode) + || s_register_operand (operands[1], SImode))" ## gcc/testsuite/gcc.target/arm/crypto-vsha1cq_u32.c ## -@@ gcc/testsuite/gcc.target/arm/crypto-vsha1cq_u32.c: TEST_SHA1C_VEC_SELECT (GET_LANE) +@@ gcc/testsuite/gcc.target/arm/crypto-vsha1cq_u32.c: uint32_t foo (void) + TEST_SHA1C_VEC_SELECT (GET_LANE) /* { dg-final { scan-assembler-times {sha1c.32\tq[0-9]+, q[0-9]+} 5 } } */ - /* { dg-final { scan-assembler-times {vdup.32\tq[0-9]+, (?:r[0-9]+|d[0-9]+\[[0-9]+\])} 4 { xfail { arm_thumb2 && arm_hf_eabi } } } } */ --/* { dg-final { scan-assembler-times {vmov.32\tr[0-9]+, d[0-9]+\[[0-9]+\]} 3 } } */ +-/* { dg-final { scan-assembler-times {vdup.32\tq[0-9]+, (?:r[0-9]+|d[0-9]+\[[0-9]+\])} 4 { xfail { arm_thumb2 && arm_hf_eabi } } } } */ ++/* { dg-final { scan-assembler-times {vdup.32\tq[0-9]+, (?:r[0-9]+|d[0-9]+\[[0-9]+\])} 4 } } */ ## gcc/testsuite/gcc.target/arm/crypto-vsha1h_u32.c ## -@@ gcc/testsuite/gcc.target/arm/crypto-vsha1h_u32.c: TEST_SHA1H_VEC_SELECT (GET_LANE) +@@ gcc/testsuite/gcc.target/arm/crypto-vsha1h_u32.c: uint32_t foo (void) + TEST_SHA1H_VEC_SELECT (GET_LANE) /* { dg-final { scan-assembler-times {sha1h.32\tq[0-9]+, q[0-9]+} 5 } } */ - /* { dg-final { scan-assembler-times {vdup.32\tq[0-9]+, (?:r[0-9]+|d[0-9]+\[[0-9]+\])} 4 { xfail { arm_thumb2 && arm_hf_eabi } } } } */ --/* { dg-final { scan-assembler-times {vmov.32\tr[0-9]+, d[0-9]+\[[0-9]+\]} 3 } } */ +-/* { dg-final { scan-assembler-times {vdup.32\tq[0-9]+, (?:r[0-9]+|d[0-9]+\[[0-9]+\])} 4 { xfail { arm_thumb2 && arm_hf_eabi } } } } */ ++/* { dg-final { scan-assembler-times {vdup.32\tq[0-9]+, (?:r[0-9]+|d[0-9]+\[[0-9]+\])} 4 } } */ ## gcc/testsuite/gcc.target/arm/crypto-vsha1mq_u32.c ## -@@ gcc/testsuite/gcc.target/arm/crypto-vsha1mq_u32.c: TEST_SHA1M_VEC_SELECT (GET_LANE) +@@ gcc/testsuite/gcc.target/arm/crypto-vsha1mq_u32.c: uint32_t foo (void) + TEST_SHA1M_VEC_SELECT (GET_LANE) /* { dg-final { scan-assembler-times {sha1m.32\tq[0-9]+, q[0-9]+} 5 } } */ - /* { dg-final { scan-assembler-times {vdup.32\tq[0-9]+, (?:r[0-9]+|d[0-9]+\[[0-9]+\])} 4 { xfail { arm_thumb2 && arm_hf_eabi } } } } */ --/* { dg-final { scan-assembler-times {vmov.32\tr[0-9]+, d[0-9]+\[[0-9]+\]} 3 } } */ +-/* { dg-final { scan-assembler-times {vdup.32\tq[0-9]+, (?:r[0-9]+|d[0-9]+\[[0-9]+\])} 4 { xfail { arm_thumb2 && arm_hf_eabi } } } } */ ++/* { dg-final { scan-assembler-times {vdup.32\tq[0-9]+, (?:r[0-9]+|d[0-9]+\[[0-9]+\])} 4 } } */ ## gcc/testsuite/gcc.target/arm/crypto-vsha1pq_u32.c ## -@@ gcc/testsuite/gcc.target/arm/crypto-vsha1pq_u32.c: TEST_SHA1P_VEC_SELECT (GET_LANE) +@@ gcc/testsuite/gcc.target/arm/crypto-vsha1pq_u32.c: uint32_t foo (void) + TEST_SHA1P_VEC_SELECT (GET_LANE) /* { dg-final { scan-assembler-times {sha1p.32\tq[0-9]+, q[0-9]+} 5 } } */ - /* { dg-final { scan-assembler-times {vdup.32\tq[0-9]+, (?:r[0-9]+|d[0-9]+\[[0-9]+\])} 4 { xfail { arm_thumb2 && arm_hf_eabi } } } } */ --/* { dg-final { scan-assembler-times {vmov.32\tr[0-9]+, d[0-9]+\[[0-9]+\]} 3 } } */ +-/* { dg-final { scan-assembler-times {vdup.32\tq[0-9]+, (?:r[0-9]+|d[0-9]+\[[0-9]+\])} 4 { xfail { arm_thumb2 && arm_hf_eabi } } } } */ ++/* { dg-final { scan-assembler-times {vdup.32\tq[0-9]+, (?:r[0-9]+|d[0-9]+\[[0-9]+\])} 4 } } */ -- 2.54.0