[x86 SSE PATCH v2] PR target/126619: Improve V4SF vector initialization.
"Roger Sayle" <[email protected]>
| Newsgroups | gmane.comp.gcc.patches |
|---|---|
| Message-ID | <[email protected]> |
Hi Hongtao,
Here's a revised version of my previous patch for PR target/126619,
incorporating your feedback and suggestions. Although x86 SSE vector
initialization is now much improved, it's not yet perfect. I'd like
to propose performing V16QI initialization improvements in a follow-up
patch. Likewise, this revised patch has one test suite regression on
IA32 caused by a difference in register allocation (unrelated to V4SF)
that can be fixed by a small improvement to the STV pass.
One thing I'd like to point out at this stage is that use of insertps
and/or pinsr chains is not just an optimization for size (fewer insns),
but these chains also use significantly fewer registers, and theoretically
on modern CPUs do not (need to) have dependency chains. A sequence of
insertps/pinsr may be executed/scheduled out-of-order [given dependency
tracking by vector element rather than by entire register]. If Intel
and AMD don't already do this, it would be a good use for a few extra
transistors. I suspect GCC's own schedulers could also be tweaked.
This patch has been tested on x86_64-pc-linux-gnu with make bootstrap
and make -k check, both with and without --target_board=unix{-m32}
with a single new failure (with -m32) of gcc.target/i386/sse2-stv-1.c,
that I'll fix shortly. Hongtao's suggestion to tweak register
preferencing (to use ? instead of !) isn't at fault/wrong, it just
exposes a latent inefficiency elsewhere. Ok for mainline?
2026-08-20 Roger Sayle <[email protected]>
Hongtao Liu <[email protected]>
gcc/ChangeLog
PR target/126619
* config/i386/i386-expand.cc
(ix86_expand_vector_init_one_nonzero) <case E_V4SFmode>:
Improved initialization of one non-zero element V4SF vectors.
(ix86_expand_vector_init_v4sf): Reuse the above function
ix86_expand_vector_init_one_nonzero where possible. Various
improvements. Fall back to using (the original)
ix86_expand_vector_init_concat in the general case, when
SSE 4.1 instructions aren't available.
* config/i386/sse.md (vec_set<mode>_0): Prefer to avoid
inter-unit moves to general purpose registers in reload.
(vec_setv4sf_sse4_1): Remove asterisk to expose to i386-expand.
(sse4_1_insertps_<mode>_init): Variant of insertps that clears
all the other elements of the destination to zero.
(sse2_insertps_v4sf_3): Implementation of the above instruction
available on SSE2 by using the pslldq instruction.
gcc/testsuite/ChangeLog
PR target/126619
* gcc.target/i386/avx-init-v4sf-1.c: Update test case.
* gcc.target/i386/avx-init-v4sf-2.c: Likewise.
* gcc.target/i386/avx2-init-v4sf-1.c: Likewise.
* gcc.target/i386/sse-init-v4sf-2.c: Likewise.
* gcc.target/i386/sse-init-v4sf-3.c: Likewise.
* gcc.target/i386/sse2-init-v4sf-1.c: Likewise.
* gcc.target/i386/sse2-init-v4sf-2.c: Likewise.
* gcc.target/i386/sse4_1-init-v4sf-2.c: Likewise.
* gcc.target/i386/sse4_1-init-v4sf-3.c: Likewise.
Thanks again,
Roger
> -----Original Message-----
> From: Hongtao Liu <[email protected]>
> Sent: 11 August 2026 08:10
> To: Roger Sayle <[email protected]>
> Cc: Patches GCC <[email protected]>; Liu, Hongtao
> <[email protected]>; Uros Bizjak <[email protected]>
> Subject: Re: [x86 SSE PATCH] PR target/126619: Improve V4SF vector
> initialization.
>
> On Sun, Aug 9, 2026 at 6:42 PM Roger Sayle <[email protected]>
> wrote:
> >
> >
> > This patch addresses PR target/126619, a performance regression caused
> > by my recent update of SSE vector initialization on x86_64. The new
> > idiom attempts to benefit from the implicit zero extension provided by
> > movss and movd, but this causes problems for (V4SF) cases were the
> > (SFmode) value is already in a register. Without newer extensions,
> > GCC's approach to zero extension (i.e. vec_init of (V4SF){ x, 0, 0, 0
> > }) is to perform an interunit transfer to a general purpose integer
> > register, and then transfer the value back again. Inter-unit moves
> > are expensive, especially on older microarchitectures.
> >
> > The problem is fixed in several ways. The first is to tweak register
> > preferencing in vec_set<mode>_0's define_insn, so that general
> > registers are only used if the source/destination is already an integer GPR.
> > This changes reload from generating (two instructions):
> >
> > movd %xmm0, %eax
> > movd %eax, %xmm0
> >
> > to instead using:
> >
> > pxor %xmm1, %xmm1
> > movss %xmm0, %xmm1
> > movaps %xmm1, %xmm0
> >
> > which requires 3 instructions, and 1 extra register, but requires no
> > inter-unit moves. This matches what clang/llvm does.
> >
> > However, it's possible to do better, borrowing an idiom from the
> > middle-end's expansion of integer zero-extensions.
> >
> > pslldq $12, %xmm0
> > psrldq $12, %xmm0
> >
> > uses two instructions, and doesn't require an extra register.
> > Indeed, initializing the vector (V4SF){ 0, 0, 0, x } can be done in a
> > single instruction, as it doesn't require a "right"
> > shift.
> >
> > Additionally, for cases such as (V4SF){ a, b, c, d }, where there is
> > no benefit from zero extension, we should continue using GCC's
> > original CONCAT of V2SF idiom, avoiding any overhead of zero extension
> > (Hongtao's suggestion in the Bugzilla PR).
> >
> > Additionally, there are some additional V4SF initialization tweaks.
> > When loading from memory, where zero extension is free "onevar_perm"s
> > should construct { x, 0, 0, 0 } then perform a shuffle using shufps,
> > but when the source is a register, it should construct { 0, 0, 0, x }
> > (using the single shift instruction described above), and perform a
> > modified shuffle using shufps from there.
> >
> > With TARGET_SSE4_1, the first insertps can be used to clear
> > (initialize) all the other elements to zero, and the remaining
> > non-zero elements can be inserted with regular insertps.
> >
> I think even with sse4.1 and above we should still use the original
> ix86_expand_vector_concat solution since insertps have sequential dependence
> with depth of 4, but vec_concat only with depth 2.
> so the vector_concat solution should still be faster(including the 4-memory case),
> maybe for optimization for size, insertps is prefered.
>
> >- else
> >+ else if (TARGET_SSE4_1)
>
> So may just
> else if (TARGET_SSE4_1 && optimize_insn_for_size_p ())
> {
> ......
> }
> else
> ix86_expand_vector_init_concat (V4SFmode, target, ops, 4);
>
> >+ {
> >+ int i;
> >+ rtx tmp = gen_reg_rtx (V4SFmode);
> >+ bool first_p = true;
> >...
> >+ else
> >+ ix86_expand_vector_init_concat (V4SFmode, target, ops, 4);
> >}
>
> > ;; see comment above inline_secondary_memory_needed function in
> >i386.cc (define_insn "vec_set<mode>_0"
> > [(set (match_operand:VI4F_128 0 "nonimmediate_operand"
> >- "=Yr,*x,v,v,v,v,x,x,v,Yr ,?x ,x ,m ,m ,m")
> >+ "=Yr,*x,v,v,v,!v,x,x,v,Yr ,?x ,x ,m ,m ,m")
> (vec_merge:VI4F_128
>
> Add *?* to the corresponding alternative *r* of operands[2] instead of add *!* to
> *v* of operands[0]?
> my experience is ?r is better to help LRA for register allocation choice.
>
> >+;; Use sse4_1_insertps_v4s[if] to vector_init one non-zero value.
> >+(define_insn "sse4_1_insertps_<mode>_init"
> >+ [(set (match_operand:VI4F_128 0 "register_operand" "=Yr,*x,v")
> >+ (vec_merge:VI4F_128
> >+ (vec_duplicate:VI4F_128
> >+ (match_operand:<ssescalarmode> 1 "nonimmediate_operand"
> "Yrjm,*xjm,vm")
> >+ (match_operand:VI4F_128 2 "const0_operand")
> >+ (match_operand:SI 3 "const248_operand")))]
> >+ "TARGET_SSE4_1"
> >+{
> >+ int op3 = INTVAL (operands[3]);
> >+ operands[3] = GEN_INT ((exact_log2 (op3) << 4) + (op3 ^ 15));
> >+ switch (which_alternative)
> >+ {
> >+ case 0:
> >+ case 1:
> >+ return "insertps\t{%3, %1, %0|%0, %1, %3}";
> >+ case 2:
> >+ return "vinsertps\t{%3, %1, %0, %0|%0, %0, %1, %3}";
>
> Split alternative 2 into v,m and use {%3, %1, %1, %0|%0, %1, %1, %3} for
> alternative *v*, it can avoid false dependence for operands[0].
> Also for the new *m* alternative, can we only enable it when not
> preferred_for_speed since there's false dependence.
> (set (attr "preferred_for_speed")
> (cond [(eq_attr "alternative" "3")
> (symbol_ref "false")]
> (symbol_ref "true")))
>
> >+ default:
> >+ gcc_unreachable ();
> >+ }
>
> >+ if (TARGET_SSE4_1)
> >+ {
> >+ if (!REG_P (var) && !MEM_P (var))
> >+ var = force_reg (SFmode, var);
>
> - rtx tmp1 = gen_reg_rtx (V4SFmode);
> - vars[0] = ops[0];
> - vars[1] = CONST0_RTX (SFmode);
> - vars[2] = CONST0_RTX (SFmode);
> - vars[3] = CONST0_RTX (SFmode);
> - ix86_expand_vector_init_v4sf (tmp1, vars);
> + if (TARGET_SSE4_1)
> + {
> + rtx tmp = gen_reg_rtx (V4SFmode);
> + rtx val = ops[0];
> + if (!REG_P (val) && !MEM_P (val))
> + val = force_reg (SFmode, val);
> + emit_insn (gen_vec_setv4sf_0 (tmp, CONST0_RTX (V4SFmode), val));
> + val = ops[1];
> + if (!REG_P (val) && !MEM_P (val))
> + val = force_reg (SFmode, val);
> + emit_insn (gen_vec_setv4sf_sse4_1 (target, tmp, val, GEN_INT (2)));
> + }
>
> I think we can generate 1 insertps for {a,b,0,0} {a, 0, b,0}, {a, 0, 0, b}, here still
> generates 2 insertps, maybe we can directy gen_sse4_1_insertps_v4sf, use
> subreg to convert SFmode to V4SFmode?
>
>
> BR,
> Hongtao
patchvi8.txt
(text/plain, 19.8 KB)
diff --git a/gcc/config/i386/i386-expand.cc b/gcc/config/i386/i386-expand.cc
index fb4224af0da..a992acf4bc5 100644
--- a/gcc/config/i386/i386-expand.cc
+++ b/gcc/config/i386/i386-expand.cc
@@ -18102,21 +18102,45 @@ ix86_expand_vector_init_one_nonzero (bool mmx_ok, machine_mode mode,
}
return true;
case E_V4SFmode:
- var = force_reg (SFmode, var);
- x = gen_rtx_VEC_DUPLICATE (V4SFmode, var);
- x = gen_rtx_VEC_MERGE (V4SFmode, x, CONST0_RTX (V4SFmode), const1_rtx);
+ if (TARGET_SSE4_1)
+ {
+ if (!REG_P (var) && !MEM_P (var))
+ var = force_reg (SFmode, var);
+ if (one_var)
+ emit_insn (gen_sse4_1_insertps_v4sf_init (target, var,
+ CONST0_RTX (V4SFmode),
+ GEN_INT (1 << one_var)));
+ else
+ emit_insn (gen_vec_setv4sf_0 (target, CONST0_RTX (V4SFmode), var));
+ return true;
+ }
+ if (TARGET_SSE2 && REG_P (var))
+ {
+ if (one_var == 3)
+ emit_insn (gen_sse2_insertps_v4sf_3 (target, var,
+ CONST0_RTX (V4SFmode)));
+ else
+ {
+ rtx tmp = gen_reg_rtx (V4SFmode);
+ emit_insn (gen_sse2_insertps_v4sf_3 (tmp, var,
+ CONST0_RTX (V4SFmode)));
+ emit_insn (gen_vec_shr_v4sf (target, tmp,
+ GEN_INT ((3 - one_var) * 32)));
+ }
+ return true;
+ }
if (!one_var)
- emit_insn (gen_rtx_SET (target, x));
+ emit_insn (gen_vec_setv4sf_0 (target, CONST0_RTX (V4SFmode), var));
else if (TARGET_SSE2)
{
rtx tmp = gen_reg_rtx (V4SFmode);
- emit_insn (gen_rtx_SET (tmp, x));
+ emit_insn (gen_vec_setv4sf_0 (tmp, CONST0_RTX (V4SFmode), var));
emit_insn (gen_vec_shl_v4sf (target, tmp, GEN_INT (one_var * 32)));
}
else
{
rtx tmp = gen_reg_rtx (V4SFmode);
- emit_insn (gen_rtx_SET (tmp, x));
+ emit_insn (gen_vec_setv4sf_0 (tmp, CONST0_RTX (V4SFmode), var));
emit_insn (gen_sse_shufps_v4sf (target, tmp, tmp,
const1_rtx,
GEN_INT (one_var == 1 ? 0 : 1),
@@ -19218,12 +19242,7 @@ ix86_expand_vector_init_v4sf (rtx target, rtx *ops)
else if (ops[1] == CONST0_RTX (SFmode)
&& ops[2] == CONST0_RTX (SFmode)
&& ops[3] == CONST0_RTX (SFmode))
- {
- rtx val = ops[0];
- if (!REG_P (val) && !MEM_P (val))
- val = force_reg (SFmode, val);
- emit_insn (gen_vec_setv4sf_0 (target, CONST0_RTX (V4SFmode), val));
- }
+ ix86_expand_vector_init_one_nonzero (false, V4SFmode, target, ops[0], 0);
else if (rtx_equal_p (ops[0], ops[1])
&& rtx_equal_p (ops[0], ops[2])
&& rtx_equal_p (ops[0], ops[3]))
@@ -19241,33 +19260,100 @@ ix86_expand_vector_init_v4sf (rtx target, rtx *ops)
rtx vec = gen_rtx_CONST_VECTOR (V4SFmode, gen_rtvec_v (4, ops));
emit_move_insn (target, vec);
}
+ else if (ops[0] == CONST0_RTX (SFmode)
+ && ops[2] == CONST0_RTX (SFmode)
+ && ops[3] == CONST0_RTX (SFmode))
+ ix86_expand_vector_init_one_nonzero (false, V4SFmode, target, ops[1], 1);
+ else if (ops[0] == CONST0_RTX (SFmode)
+ && ops[1] == CONST0_RTX (SFmode)
+ && ops[3] == CONST0_RTX (SFmode))
+ ix86_expand_vector_init_one_nonzero (false, V4SFmode, target, ops[2], 2);
+ else if (ops[0] == CONST0_RTX (SFmode)
+ && ops[1] == CONST0_RTX (SFmode)
+ && ops[2] == CONST0_RTX (SFmode))
+ ix86_expand_vector_init_one_nonzero (false, V4SFmode, target, ops[3], 3);
else if (onevar_perm_p (ops, 4, perm, vars))
{
rtx tmp = gen_reg_rtx (V4SFmode);
- vars[1] = CONST0_RTX (SFmode);
- vars[2] = CONST0_RTX (SFmode);
- vars[3] = CONST0_RTX (SFmode);
- ix86_expand_vector_init_v4sf (tmp, vars);
- emit_insn (gen_sse_shufps_v4sf (target, tmp, tmp,
- GEN_INT (perm[0]),
- GEN_INT (perm[1]),
- GEN_INT (perm[2] + 4),
- GEN_INT (perm[3] + 4)));
+ if (TARGET_SSE2 && !TARGET_SSE4_1 && REG_P (ops[0]))
+ {
+ emit_insn (gen_sse2_insertps_v4sf_3 (tmp, ops[0],
+ CONST0_RTX (V4SFmode)));
+ emit_insn (gen_sse_shufps_v4sf (target, tmp, tmp,
+ GEN_INT (perm[0] ? 0 : 3),
+ GEN_INT (perm[1] ? 0 : 3),
+ GEN_INT (perm[2] ? 4 : 7),
+ GEN_INT (perm[3] ? 4 : 7)));
+ }
+ else
+ {
+ vars[1] = CONST0_RTX (SFmode);
+ vars[2] = CONST0_RTX (SFmode);
+ vars[3] = CONST0_RTX (SFmode);
+ ix86_expand_vector_init_v4sf (tmp, vars);
+ emit_insn (gen_sse_shufps_v4sf (target, tmp, tmp,
+ GEN_INT (perm[0]),
+ GEN_INT (perm[1]),
+ GEN_INT (perm[2] + 4),
+ GEN_INT (perm[3] + 4)));
+ }
}
else if (ops[2] == CONST0_RTX (SFmode)
&& ops[3] == CONST0_RTX (SFmode))
{
- rtx tmp1 = gen_reg_rtx (V4SFmode);
- vars[0] = ops[0];
- vars[1] = CONST0_RTX (SFmode);
- vars[2] = CONST0_RTX (SFmode);
- vars[3] = CONST0_RTX (SFmode);
- ix86_expand_vector_init_v4sf (tmp1, vars);
+ if (TARGET_SSE4_1)
+ {
+ rtx tmp = gen_reg_rtx (V4SFmode);
+ rtx val = ops[0];
+ if (!REG_P (val) && !MEM_P (val))
+ val = force_reg (SFmode, val);
+ emit_insn (gen_vec_setv4sf_0 (tmp, CONST0_RTX (V4SFmode), val));
+ val = ops[1];
+ if (!REG_P (val) && !MEM_P (val))
+ val = force_reg (SFmode, val);
+ emit_insn (gen_vec_setv4sf_sse4_1 (target, tmp, val, GEN_INT (2)));
+ }
+ else if (MEM_P (ops[0]) && MEM_P (ops[1]))
+ {
+ rtx tmp1 = gen_reg_rtx (V4SFmode);
+ vars[0] = ops[0];
+ vars[1] = CONST0_RTX (SFmode);
+ vars[2] = CONST0_RTX (SFmode);
+ vars[3] = CONST0_RTX (SFmode);
+ ix86_expand_vector_init_v4sf (tmp1, vars);
+ rtx tmp2 = gen_reg_rtx (V4SFmode);
+ vars[0] = ops[1];
+ ix86_expand_vector_init_v4sf (tmp2, vars);
+ emit_insn (gen_vec_interleave_lowv4sf (target, tmp1, tmp2));
+ }
+ else if (TARGET_SSE2)
+ {
+ rtx tmp = gen_reg_rtx (V2SFmode);
+ ix86_expand_vector_init_concat (V2SFmode, tmp, ops, 2);
+ tmp = gen_rtx_VEC_CONCAT (V4SFmode, tmp, CONST0_RTX (V2SFmode));
+ emit_insn (gen_rtx_SET (target, tmp));
+ }
+ else
+ {
+ rtx tmp1 = gen_reg_rtx (V2SFmode);
+ ix86_expand_vector_init_concat (V2SFmode, tmp1, ops, 2);
+ rtx tmp2 = gen_reg_rtx (V2SFmode);
+ emit_insn (gen_rtx_SET (tmp2, CONST0_RTX (V2SFmode)));
+ rtx tmp = gen_rtx_VEC_CONCAT (V4SFmode, tmp1, tmp2);
+ emit_insn (gen_rtx_SET (target, tmp));
+ }
+ }
+ else if (MEM_P (ops[0])
+ && ops[1] == CONST0_RTX (SFmode)
+ && MEM_P (ops[2])
+ && ops[3] == CONST0_RTX (SFmode))
+ {
+ rtx tmp1 = gen_reg_rtx (V4SFmode);
+ ix86_expand_vector_init_one_nonzero (false, V4SFmode, tmp1, ops[0], 0);
rtx tmp2 = gen_reg_rtx (V4SFmode);
- vars[0] = ops[1];
- ix86_expand_vector_init_v4sf (tmp2, vars);
- emit_insn (gen_vec_interleave_lowv4sf (target, tmp1, tmp2));
+ ix86_expand_vector_init_one_nonzero (false, V4SFmode, tmp2, ops[2], 0);
+ emit_insn (gen_sse_movlhps (target, tmp1, tmp2));
}
else if (twovar_perm_p (ops, 4, perm, vars))
{
@@ -19303,7 +19389,32 @@ ix86_expand_vector_init_v4sf (rtx target, rtx *ops)
emit_move_insn (tmp2, vec);
emit_insn (gen_rtx_SET (target, gen_rtx_IOR (V4SFmode, tmp1, tmp2)));
}
- else
+ else if (TARGET_SSE4_1 && optimize_insn_for_size_p ())
+ {
+ int i;
+ rtx tmp = gen_reg_rtx (V4SFmode);
+ bool first_p = true;
+ for (i = 0; i < 4; i++)
+ if (ops[i] != CONST0_RTX (SFmode))
+ {
+ rtx idx = GEN_INT (1 << i);
+ rtx val = ops[i];
+ if (!REG_P (val) && !MEM_P (val))
+ val = force_reg (SFmode, val);
+ rtx pat = first_p
+ ? gen_sse4_1_insertps_v4sf_init (tmp, val,
+ CONST0_RTX (V4SFmode),
+ idx)
+ : gen_vec_setv4sf_sse4_1 (tmp, tmp, val, idx);
+ emit_insn (pat);
+ first_p = false;
+ }
+ emit_move_insn (target, tmp);
+ }
+ else if (MEM_P (ops[0])
+ && MEM_P (ops[1])
+ && MEM_P (ops[2])
+ && MEM_P (ops[3]))
{
rtx tmp1 = gen_reg_rtx (V4SFmode);
rtx tmp2 = gen_reg_rtx (V4SFmode);
@@ -19319,6 +19430,8 @@ ix86_expand_vector_init_v4sf (rtx target, rtx *ops)
const0_rtx, const1_rtx,
GEN_INT (4), GEN_INT (5)));
}
+ else
+ ix86_expand_vector_init_concat (V4SFmode, target, ops, 4);
}
/* A subroutine of ix86_expand_vector_init for V8HImode. */
diff --git a/gcc/config/i386/sse.md b/gcc/config/i386/sse.md
index bf4c4505601..5a081d7b775 100644
--- a/gcc/config/i386/sse.md
+++ b/gcc/config/i386/sse.md
@@ -12572,11 +12572,11 @@
;; see comment above inline_secondary_memory_needed function in i386.cc
(define_insn "vec_set<mode>_0"
[(set (match_operand:VI4F_128 0 "nonimmediate_operand"
- "=Yr,*x,v,v,v,v,x,x,v,Yr ,?x ,x ,m ,m ,m")
+ "=Yr,*x,v,v,v,v ,x,x,v,Yr ,?x ,x ,m ,m ,m")
(vec_merge:VI4F_128
(vec_duplicate:VI4F_128
(match_operand:<ssescalarmode> 2 "general_operand"
- " Yr,*x,v,v,m,r ,m,x,v,?jrjm,?jrjm,?rm,!x,?re,!*fF"))
+ " Yr,*x,v,v,m,?r,m,x,v,?jrjm,?jrjm,?rm,!x,?re,!*fF"))
(match_operand:VI4F_128 1 "nonimm_or_0_operand"
" C , C,C,C,C,C ,C,0,v,0 ,0 ,x ,0 ,0 ,0")
(const_int 1)))]
@@ -12834,7 +12834,7 @@
(set_attr "mode" "HF,HI")])
;; A subset is vec_setv4sf.
-(define_insn "*vec_setv4sf_sse4_1"
+(define_insn "vec_setv4sf_sse4_1"
[(set (match_operand:V4SF 0 "register_operand" "=Yr,*x,v")
(vec_merge:V4SF
(vec_duplicate:V4SF
@@ -12998,6 +12998,62 @@
(set_attr "prefix" "orig,maybe_evex")
(set_attr "mode" "V4SF")])
+;; Use sse4_1_insertps_v4s[if] to vector_init one non-zero value.
+(define_insn "sse4_1_insertps_<mode>_init"
+ [(set (match_operand:VI4F_128 0 "register_operand" "=Yr,*x,v,v")
+ (vec_merge:VI4F_128
+ (vec_duplicate:VI4F_128
+ (match_operand:<ssescalarmode> 1 "nonimmediate_operand" "Yrjm,*xjm,v,m"))
+ (match_operand:VI4F_128 2 "const0_operand")
+ (match_operand:SI 3 "const248_operand")))]
+ "TARGET_SSE4_1"
+{
+ int op3 = INTVAL (operands[3]);
+ operands[3] = GEN_INT ((exact_log2 (op3) << 4) + (op3 ^ 15));
+ switch (which_alternative)
+ {
+ case 0:
+ case 1:
+ return "insertps\t{%3, %1, %0|%0, %1, %3}";
+ case 2:
+ return "vinsertps\t{%3, %1, %1, %0|%0, %1, %1, %3}";
+ case 3:
+ return "vinsertps\t{%3, %1, %0, %0|%0, %0, %1, %3}";
+ default:
+ gcc_unreachable ();
+ }
+}
+ [(set_attr "isa" "noavx,noavx,avx,avx")
+ (set_attr "type" "sselog")
+ (set_attr "addr" "gpr16,gpr16,*,*")
+ (set_attr "prefix_data16" "1,1,*,*")
+ (set_attr "prefix_extra" "1")
+ (set_attr "length_immediate" "1")
+ (set_attr "prefix" "orig,orig,maybe_evex,maybe_evex")
+ (set (attr "preferred_for_speed")
+ (cond [(eq_attr "alternative" "3")
+ (symbol_ref "false")]
+ (symbol_ref "true")))
+ (set_attr "mode" "V4SF")])
+
+;; Use SSE2's pslldq to vector_init v4sf one non-zero value at 3.
+(define_insn "sse2_insertps_v4sf_3"
+ [(set (match_operand:V4SF 0 "register_operand" "=x")
+ (vec_merge:V4SF
+ (vec_duplicate:V4SF
+ (match_operand:SF 1 "register_operand" "0"))
+ (match_operand:V4SF 2 "const0_operand")
+ (const_int 8)))]
+ "TARGET_SSE2 && !TARGET_SSE4_1"
+ "pslldq\t{$12, %0|%0, 12}"
+ [(set_attr "isa" "noavx")
+ (set_attr "type" "sseishft")
+ (set_attr "length_immediate" "1")
+ (set_attr "atom_unit" "sishuf")
+ (set_attr "prefix_data16" "1")
+ (set_attr "prefix" "orig")
+ (set_attr "mode" "V4SF")])
+
(define_split
[(set (match_operand:VI4F_128 0 "memory_operand")
(vec_merge:VI4F_128
diff --git a/gcc/testsuite/gcc.target/i386/avx-init-v4sf-1.c b/gcc/testsuite/gcc.target/i386/avx-init-v4sf-1.c
index a1f63c6a44a..18031805cb1 100644
--- a/gcc/testsuite/gcc.target/i386/avx-init-v4sf-1.c
+++ b/gcc/testsuite/gcc.target/i386/avx-init-v4sf-1.c
@@ -24,10 +24,9 @@ v4sf fm000() { return (v4sf){m,0.0f,0.0f,0.0f}; }
v4sf fm0m0() { return (v4sf){m,0.0f,m,0.0f}; }
v4sf fmmmm() { return (v4sf){m,m,m,m}; }
-/* { dg-final { scan-assembler-times "vinsertps" 16 } } */
-/* { dg-final { scan-assembler-times "vpslldq" 3 } } */
-/* { dg-final { scan-assembler-times "vshufps" 3 } } */
+/* { dg-final { scan-assembler-times "vbroadcastss" 1 } } */
+/* { dg-final { scan-assembler-times "vinsertps" 12 } } */
/* { dg-final { scan-assembler-times "vmovlhps" 4 } } */
-/* { dg-final { scan-assembler-times "vunpcklps" 5 } } */
/* { dg-final { scan-assembler-times "vmovss" 2 } } */
-/* { dg-final { scan-assembler-times "vbroadcastss" 1 } } */
+/* { dg-final { scan-assembler-times "vshufps" 3 } } */
+/* { dg-final { scan-assembler-times "vunpcklps" 2 } } */
diff --git a/gcc/testsuite/gcc.target/i386/avx-init-v4sf-2.c b/gcc/testsuite/gcc.target/i386/avx-init-v4sf-2.c
index 6d3ff179d16..ead89149820 100644
--- a/gcc/testsuite/gcc.target/i386/avx-init-v4sf-2.c
+++ b/gcc/testsuite/gcc.target/i386/avx-init-v4sf-2.c
@@ -24,9 +24,8 @@ v4sf fm000() { return (v4sf){m,0.0f,0.0f,0.0f}; }
v4sf fm0m0() { return (v4sf){m,0.0f,m,0.0f}; }
v4sf fmmmm() { return (v4sf){m,m,m,m}; }
-/* { dg-final { scan-assembler-times "vmovss" 18 } } */
-/* { dg-final { scan-assembler-times "vpslldq" 3 } } */
-/* { dg-final { scan-assembler-times "vshufps" 2 } } */
-/* { dg-final { scan-assembler-times "vmovlhps" 4 } } */
/* { dg-final { scan-assembler-times "vbroadcastss" 2 } } */
-/* { dg-final { scan-assembler-times "vunpcklps" 5 } } */
+/* { dg-final { scan-assembler-times "vinsertps" 8 } } */
+/* { dg-final { scan-assembler-times "vmovlhps" 4 } } */
+/* { dg-final { scan-assembler-times "vmovss" 13 } } */
+/* { dg-final { scan-assembler-times "vshufps" 2 } } */
diff --git a/gcc/testsuite/gcc.target/i386/avx2-init-v4sf-1.c b/gcc/testsuite/gcc.target/i386/avx2-init-v4sf-1.c
index 23ff0e76836..2dcdd608c7c 100644
--- a/gcc/testsuite/gcc.target/i386/avx2-init-v4sf-1.c
+++ b/gcc/testsuite/gcc.target/i386/avx2-init-v4sf-1.c
@@ -24,10 +24,9 @@ v4sf fm000() { return (v4sf){m,0.0f,0.0f,0.0f}; }
v4sf fm0m0() { return (v4sf){m,0.0f,m,0.0f}; }
v4sf fmmmm() { return (v4sf){m,m,m,m}; }
-/* { dg-final { scan-assembler-times "vinsertps" 16 } } */
-/* { dg-final { scan-assembler-times "vpslldq" 3 } } */
-/* { dg-final { scan-assembler-times "vshufps" 2 } } */
+/* { dg-final { scan-assembler-times "vbroadcastss" 2 } } */
+/* { dg-final { scan-assembler-times "vinsertps" 12 } } */
/* { dg-final { scan-assembler-times "vmovlhps" 4 } } */
-/* { dg-final { scan-assembler-times "vunpcklps" 5 } } */
/* { dg-final { scan-assembler-times "vmovss" 2 } } */
-/* { dg-final { scan-assembler-times "vbroadcastss" 2 } } */
+/* { dg-final { scan-assembler-times "vshufps" 2 } } */
+/* { dg-final { scan-assembler-times "vunpcklps" 2 } } */
diff --git a/gcc/testsuite/gcc.target/i386/sse-init-v4sf-2.c b/gcc/testsuite/gcc.target/i386/sse-init-v4sf-2.c
index c958ae9e7f8..086ddbbc2a6 100644
--- a/gcc/testsuite/gcc.target/i386/sse-init-v4sf-2.c
+++ b/gcc/testsuite/gcc.target/i386/sse-init-v4sf-2.c
@@ -24,9 +24,9 @@ v4sf fm000() { return (v4sf){m,0.0f,0.0f,0.0f}; }
v4sf fm0m0() { return (v4sf){m,0.0f,m,0.0f}; }
v4sf fmmmm() { return (v4sf){m,m,m,m}; }
-/* { dg-final { scan-assembler-times "xorps" 16 } } */
-/* { dg-final { scan-assembler-times "movss" 19 } } */
-/* { dg-final { scan-assembler-times "movaps" 10 } } */
+/* { dg-final { scan-assembler-times "xorps" 9 } } */
+/* { dg-final { scan-assembler-times "movss" 9 } } */
+/* { dg-final { scan-assembler-times "movaps" 6 } } */
/* { dg-final { scan-assembler-times "shufps" 7 } } */
-/* { dg-final { scan-assembler-times "movlhps" 4 } } */
+/* { dg-final { scan-assembler-times "movlhps" 7 } } */
/* { dg-final { scan-assembler-times "unpcklps" 5 } } */
diff --git a/gcc/testsuite/gcc.target/i386/sse-init-v4sf-3.c b/gcc/testsuite/gcc.target/i386/sse-init-v4sf-3.c
index d98386e9ffc..ad1777c476b 100644
--- a/gcc/testsuite/gcc.target/i386/sse-init-v4sf-3.c
+++ b/gcc/testsuite/gcc.target/i386/sse-init-v4sf-3.c
@@ -24,7 +24,8 @@ v4sf fm000() { return (v4sf){m,0.0f,0.0f,0.0f}; }
v4sf fm0m0() { return (v4sf){m,0.0f,m,0.0f}; }
v4sf fmmmm() { return (v4sf){m,m,m,m}; }
+/* { dg-final { scan-assembler-times "movlhps" 7 } } */
/* { dg-final { scan-assembler-times "movss" 20 } } */
/* { dg-final { scan-assembler-times "shufps" 7 } } */
-/* { dg-final { scan-assembler-times "movlhps" 4 } } */
/* { dg-final { scan-assembler-times "unpcklps" 5 } } */
+/* { dg-final { scan-assembler-times "xorps" 3 } } */
diff --git a/gcc/testsuite/gcc.target/i386/sse2-init-v4sf-1.c b/gcc/testsuite/gcc.target/i386/sse2-init-v4sf-1.c
index 716d55b8161..9fb63dfce5a 100644
--- a/gcc/testsuite/gcc.target/i386/sse2-init-v4sf-1.c
+++ b/gcc/testsuite/gcc.target/i386/sse2-init-v4sf-1.c
@@ -24,9 +24,10 @@ v4sf fm000() { return (v4sf){m,0.0f,0.0f,0.0f}; }
v4sf fm0m0() { return (v4sf){m,0.0f,m,0.0f}; }
v4sf fmmmm() { return (v4sf){m,m,m,m}; }
-/* { dg-final { scan-assembler-times "movd" 32 } } */
-/* { dg-final { scan-assembler-times "pslldq" 3 } } */
-/* { dg-final { scan-assembler-times "shufps" 4 } } */
-/* { dg-final { scan-assembler-times "movlhps" 4 } } */
+/* { dg-final { scan-assembler-times "pslldq" 7 } } */
+/* { dg-final { scan-assembler-times "psrldq" 3 } } */
+/* { dg-final { scan-assembler-times "shufps" 6 } } */
+/* { dg-final { scan-assembler-times "movlhps" 2 } } */
/* { dg-final { scan-assembler-times "unpcklps" 5 } } */
+/* { dg-final { scan-assembler-times "movq" 3 } } */
/* { dg-final { scan-assembler-times "movss" 3 } } */
diff --git a/gcc/testsuite/gcc.target/i386/sse2-init-v4sf-2.c b/gcc/testsuite/gcc.target/i386/sse2-init-v4sf-2.c
index 44ed9944ef2..5e55ce14f49 100644
--- a/gcc/testsuite/gcc.target/i386/sse2-init-v4sf-2.c
+++ b/gcc/testsuite/gcc.target/i386/sse2-init-v4sf-2.c
@@ -24,8 +24,10 @@ v4sf fm000() { return (v4sf){m,0.0f,0.0f,0.0f}; }
v4sf fm0m0() { return (v4sf){m,0.0f,m,0.0f}; }
v4sf fmmmm() { return (v4sf){m,m,m,m}; }
-/* { dg-final { scan-assembler-times "movss" 20 } } */
-/* { dg-final { scan-assembler-times "pslldq" 3 } } */
-/* { dg-final { scan-assembler-times "shufps" 4 } } */
-/* { dg-final { scan-assembler-times "movlhps" 4 } } */
+/* { dg-final { scan-assembler-times "pslldq" 7 } } */
+/* { dg-final { scan-assembler-times "psrldq" 3 } } */
+/* { dg-final { scan-assembler-times "shufps" 6 } } */
+/* { dg-final { scan-assembler-times "movlhps" 2 } } */
/* { dg-final { scan-assembler-times "unpcklps" 5 } } */
+/* { dg-final { scan-assembler-times "movq" 3 } } */
+/* { dg-final { scan-assembler-times "movss" 20 } } */
diff --git a/gcc/testsuite/gcc.target/i386/sse4_1-init-v4sf-2.c b/gcc/testsuite/gcc.target/i386/sse4_1-init-v4sf-2.c
index b4e7f7293a9..90f0fbbd04a 100644
--- a/gcc/testsuite/gcc.target/i386/sse4_1-init-v4sf-2.c
+++ b/gcc/testsuite/gcc.target/i386/sse4_1-init-v4sf-2.c
@@ -24,9 +24,9 @@ v4sf fm000() { return (v4sf){m,0.0f,0.0f,0.0f}; }
v4sf fm0m0() { return (v4sf){m,0.0f,m,0.0f}; }
v4sf fmmmm() { return (v4sf){m,m,m,m}; }
-/* { dg-final { scan-assembler-times "insertps" 16 } } */
-/* { dg-final { scan-assembler-times "pslldq" 3 } } */
-/* { dg-final { scan-assembler-times "shufps" 4 } } */
+/* { dg-final { scan-assembler-times "insertps" 12 } } */
/* { dg-final { scan-assembler-times "movlhps" 4 } } */
-/* { dg-final { scan-assembler-times "unpcklps" 5 } } */
/* { dg-final { scan-assembler-times "movss" 3 } } */
+/* { dg-final { scan-assembler-not "pslldq" } } */
+/* { dg-final { scan-assembler-times "shufps" 4 } } */
+/* { dg-final { scan-assembler-times "unpcklps" 2 } } */
diff --git a/gcc/testsuite/gcc.target/i386/sse4_1-init-v4sf-3.c b/gcc/testsuite/gcc.target/i386/sse4_1-init-v4sf-3.c
index 9a51d1b7dcf..c6e6308a7e8 100644
--- a/gcc/testsuite/gcc.target/i386/sse4_1-init-v4sf-3.c
+++ b/gcc/testsuite/gcc.target/i386/sse4_1-init-v4sf-3.c
@@ -24,8 +24,9 @@ v4sf fm000() { return (v4sf){m,0.0f,0.0f,0.0f}; }
v4sf fm0m0() { return (v4sf){m,0.0f,m,0.0f}; }
v4sf fmmmm() { return (v4sf){m,m,m,m}; }
-/* { dg-final { scan-assembler-times "movss" 20 } } */
-/* { dg-final { scan-assembler-times "pslldq" 3 } } */
-/* { dg-final { scan-assembler-times "shufps" 4 } } */
+/* { dg-final { scan-assembler-times "insertps" 8 } } */
+/* { dg-final { scan-assembler-times "movss" 12 } } */
/* { dg-final { scan-assembler-times "movlhps" 4 } } */
-/* { dg-final { scan-assembler-times "unpcklps" 5 } } */
+/* { dg-final { scan-assembler-not "pslldq" } } */
+/* { dg-final { scan-assembler-times "shufps" 4 } } */
+/* { dg-final { scan-assembler-not "unpcklps" } } */