Re: [RFC] aarch64: Split selected LDPs to improve store forwarding

On 16/08/2026 20:51, Andrea Pinski wrote:
> On Sun, Aug 16, 2026 at 8:17 PM huzife <[email protected]> wrote:
> >
> > A load pair that partially overlaps a recent scalar store can prevent the
> > load from using store-to-load forwarding.  Splitting the pair lets the
> > overlapping half use the forwarding path.
> >
> > Add a late AArch64 RTL pass that handles non-writeback SI and DI GPR load
> > pairs.  Search a bounded part of the containing basic block for a scalar
> > store to either half, and replace the pair only when both scalar loads are
> > recognized.  Reject volatile, frame-related and exception-sensitive forms,
> > and disable the transformation for size optimization.  Keep the pass off by
> > default behind a target parameter while its profitability is evaluated.
> 
> 
> I am not sure we want this.  Is this dependent on the micro-arch? Can
> you expand on which micro-arch where this helps out?
> Also shouldn't instead of a separate pass which undoes what an earlier
> pass. Won't it be better to improve the heuristics of ldp_fusion pass
> to reject this from happening in the first place for these cases?

I agree with everything Andrea said here, it would be good to know which
u-arches are expected to benefit, and potentially which workloads show a
benefit with this transformation.

If we do decide to to do this, I agree that it would be better to try
and prevent the pairs from being formed in the first place (i.e. in
pair-fusion) if possible.

Thanks,
Alex

> 
> >
> > gcc/ChangeLog:
> >
> >         * config.gcc (aarch64*-*-*): Add aarch64-ldp-split.o.
> >         * config/aarch64/aarch64-ldp-split.cc: New file.
> >         * config/aarch64/aarch64-passes.def: Insert pass_split_ldp_stlf before
> >         pass_sched2.
> >         * config/aarch64/aarch64-protos.h (make_pass_split_ldp_stlf): Declare.
> >         * config/aarch64/aarch64.opt: Add aarch64-split-ldp-stlf parameter.
> >         * config/aarch64/t-aarch64 (aarch64-ldp-split.o): New rule.
> >         * doc/params.texi (aarch64-split-ldp-stlf): Document.
> >
> > gcc/testsuite/ChangeLog:
> >
> >         * gcc.target/aarch64/split-ldp-stlf-1.c: New test.
> >         * gcc.target/aarch64/split-ldp-stlf-2.c: New test.
> >         * gcc.target/aarch64/split-ldp-stlf-3.c: New test.
> >         * gcc.target/aarch64/split-ldp-stlf-4.c: New test.
> >         * gcc.target/aarch64/split-ldp-stlf-5.c: New test.
> >         * gcc.target/aarch64/split-ldp-stlf-6.c: New test.
> >
> > Signed-off-by: huzife <[email protected]>
> > ---
> >  gcc/config.gcc                                |   1 +
> >  gcc/config/aarch64/aarch64-ldp-split.cc       | 316 ++++++++++++++++++
> >  gcc/config/aarch64/aarch64-passes.def         |   1 +
> >  gcc/config/aarch64/aarch64-protos.h           |   1 +
> >  gcc/config/aarch64/aarch64.opt                |   5 +
> >  gcc/config/aarch64/t-aarch64                  |   6 +
> >  gcc/doc/params.texi                           |   8 +
> >  .../gcc.target/aarch64/split-ldp-stlf-1.c     |  43 +++
> >  .../gcc.target/aarch64/split-ldp-stlf-2.c     |  13 +
> >  .../gcc.target/aarch64/split-ldp-stlf-3.c     |  22 ++
> >  .../gcc.target/aarch64/split-ldp-stlf-4.c     |  14 +
> >  .../gcc.target/aarch64/split-ldp-stlf-5.c     |  13 +
> >  .../gcc.target/aarch64/split-ldp-stlf-6.c     |  13 +
> >  13 files changed, 456 insertions(+)
> >  create mode 100644 gcc/config/aarch64/aarch64-ldp-split.cc
> >  create mode 100644 gcc/testsuite/gcc.target/aarch64/split-ldp-stlf-1.c
> >  create mode 100644 gcc/testsuite/gcc.target/aarch64/split-ldp-stlf-2.c
> >  create mode 100644 gcc/testsuite/gcc.target/aarch64/split-ldp-stlf-3.c
> >  create mode 100644 gcc/testsuite/gcc.target/aarch64/split-ldp-stlf-4.c
> >  create mode 100644 gcc/testsuite/gcc.target/aarch64/split-ldp-stlf-5.c
> >  create mode 100644 gcc/testsuite/gcc.target/aarch64/split-ldp-stlf-6.c
> >
> > diff --git a/gcc/config.gcc b/gcc/config.gcc
> > index 0477bf22c8a..00aa965f162 100644
> > --- a/gcc/config.gcc
> > +++ b/gcc/config.gcc
> > @@ -362,6 +362,7 @@ aarch64*-*-*)
> >         extra_objs="${extra_objs} aarch-bti-insert.o"
> >         extra_objs="${extra_objs} aarch64-early-ra.o"
> >         extra_objs="${extra_objs} aarch64-ldp-fusion.o"
> > +       extra_objs="${extra_objs} aarch64-ldp-split.o"
> >         extra_objs="${extra_objs} aarch64-sched-dispatch.o"
> >         extra_objs="${extra_objs} aarch64-json-tunings-printer.o"
> >         extra_objs="${extra_objs} aarch64-json-tunings-parser.o"
> > diff --git a/gcc/config/aarch64/aarch64-ldp-split.cc b/gcc/config/aarch64/aarch64-ldp-split.cc
> > new file mode 100644
> > index 00000000000..a4c2c159529
> > --- /dev/null
> > +++ b/gcc/config/aarch64/aarch64-ldp-split.cc
> > @@ -0,0 +1,316 @@
> > +/* Split selected load pairs to improve store-to-load forwarding on AArch64.
> > +   Copyright (C) 2026 Free Software Foundation, Inc.
> > +
> > +   This file is part of GCC.
> > +
> > +   GCC is free software; you can redistribute it and/or modify it under
> > +   the terms of the GNU General Public License as published by the Free
> > +   Software Foundation; either version 3, or (at your option) any later
> > +   version.
> > +
> > +   GCC is distributed in the hope that it will be useful, but WITHOUT ANY
> > +   WARRANTY; without even the implied warranty of MERCHANTABILITY or
> > +   FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
> > +   for more details.
> > +
> > +   You should have received a copy of the GNU General Public License
> > +   along with GCC; see the file COPYING3.  If not see
> > +   <http://www.gnu.org/licenses/>.  */
> > +
> > +#define IN_TARGET_CODE 1
> > +
> > +#include "config.h"
> > +#include "system.h"
> > +#include "coretypes.h"
> > +#include "backend.h"
> > +#include "rtl.h"
> > +#include "df.h"
> > +#include "memmodel.h"
> > +#include "emit-rtl.h"
> > +#include "tree-pass.h"
> > +#include "insn-attr.h"
> > +#include "insn-constants.h"
> > +#include "insn-config.h"
> > +#include "recog.h"
> > +#include "cfgrtl.h"
> > +#include "predict.h"
> > +
> > +namespace {
> > +
> > +/* A non-writeback GPR load pair that is safe for this pass to split.  */
> > +struct ldp_info
> > +{
> > +  rtx dests[2];
> > +  rtx mem;
> > +  rtx base;
> > +  machine_mode mode;
> > +  HOST_WIDE_INT offset;
> > +  HOST_WIDE_INT access_size;
> > +};
> > +
> > +/* Return true if REG is a hard general-purpose register in MODE.  */
> > +
> > +static bool
> > +gpr_operand_p (rtx reg, machine_mode mode)
> > +{
> > +  return (REG_P (reg)
> > +         && HARD_REGISTER_P (reg)
> > +         && GP_REGNUM_P (REGNO (reg))
> > +         && GET_MODE (reg) == mode);
> > +}
> > +
> > +/* Extract the MEM from one half of an AArch64 load pair.  */
> > +
> > +static rtx
> > +extract_ldp_mem (rtx set, rtx dest, int unspec_code)
> > +{
> > +  if (GET_CODE (set) != SET || !rtx_equal_p (SET_DEST (set), dest))
> > +    return NULL_RTX;
> > +
> > +  rtx src = SET_SRC (set);
> > +  if (GET_CODE (src) != UNSPEC
> > +      || XINT (src, 1) != unspec_code
> > +      || XVECLEN (src, 0) != 1)
> > +    return NULL_RTX;
> > +
> > +  rtx mem = XVECEXP (src, 0, 0);
> > +  return MEM_P (mem) ? mem : NULL_RTX;
> > +}
> > +
> > +/* If INSN is an ordinary, non-writeback SI or DI GPR LDP, describe it in
> > +   INFO and return true.  Deliberately reject FP/SIMD, TImode, volatile,
> > +   frame-related, and EH-sensitive forms.  */
> > +
> > +static bool
> > +extract_ldp (rtx_insn *insn, ldp_info *info)
> > +{
> > +  if (!NONDEBUG_INSN_P (insn)
> > +      || INSN_CODE (insn) < 0
> > +      || get_attr_ldpstp (insn) != LDPSTP_LDP
> > +      || RTX_FRAME_RELATED_P (insn)
> > +      || find_reg_note (insn, REG_EH_REGION, NULL_RTX))
> > +    return false;
> > +
> > +  rtx pattern = PATTERN (insn);
> > +  if (GET_CODE (pattern) != PARALLEL || XVECLEN (pattern, 0) != 2)
> > +    return false;
> > +
> > +  rtx sets[2] = { XVECEXP (pattern, 0, 0), XVECEXP (pattern, 0, 1) };
> > +  if (GET_CODE (sets[0]) != SET || GET_CODE (sets[1]) != SET)
> > +    return false;
> > +
> > +  info->dests[0] = SET_DEST (sets[0]);
> > +  info->dests[1] = SET_DEST (sets[1]);
> > +  info->mode = GET_MODE (info->dests[0]);
> > +  if ((info->mode != SImode && info->mode != DImode)
> > +      || !gpr_operand_p (info->dests[0], info->mode)
> > +      || !gpr_operand_p (info->dests[1], info->mode)
> > +      || reg_overlap_mentioned_p (info->dests[0], info->dests[1]))
> > +    return false;
> > +
> > +  rtx mem0 = extract_ldp_mem (sets[0], info->dests[0], UNSPEC_LDP_FST);
> > +  rtx mem1 = extract_ldp_mem (sets[1], info->dests[1], UNSPEC_LDP_SND);
> > +  if (!mem0
> > +      || !mem1
> > +      || !rtx_equal_p (mem0, mem1)
> > +      || MEM_VOLATILE_P (mem0)
> > +      || side_effects_p (XEXP (mem0, 0))
> > +      || !MEM_SIZE_KNOWN_P (mem0)
> > +      || !MEM_SIZE (mem0).is_constant ())
> > +    return false;
> > +
> > +  info->access_size = GET_MODE_SIZE (info->mode).to_constant ();
> > +  if (!known_eq (MEM_SIZE (mem0), 2 * info->access_size))
> > +    return false;
> > +
> > +  poly_int64 offset;
> > +  info->base = strip_offset (XEXP (mem0, 0), &offset);
> > +  if (!REG_P (info->base) || !offset.is_constant (&info->offset))
> > +    return false;
> > +
> > +  info->mem = mem0;
> > +  return true;
> > +}
> > +
> > +/* Return true if INSN is a nonvolatile GPR store of one LDP half, using the
> > +   same unmodified base as INFO and writing exactly the low or high half.  */
> > +
> > +static bool
> > +forwarding_store_p (rtx_insn *insn, const ldp_info &info)
> > +{
> > +  if (!NONDEBUG_INSN_P (insn) || RTX_FRAME_RELATED_P (insn))
> > +    return false;
> > +
> > +  rtx set = single_set (insn);
> > +  if (!set || !MEM_P (SET_DEST (set)))
> > +    return false;
> > +
> > +  rtx mem = SET_DEST (set);
> > +  rtx value = SET_SRC (set);
> > +  if (GET_MODE (mem) != info.mode
> > +      || !gpr_operand_p (value, info.mode)
> > +      || MEM_VOLATILE_P (mem)
> > +      || side_effects_p (XEXP (mem, 0))
> > +      || !MEM_SIZE_KNOWN_P (mem)
> > +      || !known_eq (MEM_SIZE (mem), info.access_size))
> > +    return false;
> > +
> > +  poly_int64 poly_offset;
> > +  rtx base = strip_offset (XEXP (mem, 0), &poly_offset);
> > +  HOST_WIDE_INT offset;
> > +  if (!REG_P (base)
> > +      || !rtx_equal_p (base, info.base)
> > +      || !poly_offset.is_constant (&offset))
> > +    return false;
> > +
> > +  return (offset == info.offset
> > +         || offset == info.offset + info.access_size);
> > +}
> > +
> > +/* Search backwards in the containing basic block for a nearby store that
> > +   makes splitting PAIR profitable.  Stop once the base value changes.  */
> > +
> > +static rtx_insn *
> > +find_forwarding_store (rtx_insn *pair, const ldp_info &info)
> > +{
> > +  basic_block bb = BLOCK_FOR_INSN (pair);
> > +  unsigned int distance = 0;
> > +  unsigned int max_distance = (unsigned int) aarch64_split_ldp_stlf;
> > +
> > +  for (rtx_insn *insn = PREV_INSN (pair);
> > +       insn && BLOCK_FOR_INSN (insn) == bb;
> > +       insn = PREV_INSN (insn))
> > +    {
> > +      if (!NONDEBUG_INSN_P (insn))
> > +       continue;
> > +
> > +      if (++distance > max_distance)
> > +       break;
> > +
> > +      if (CALL_P (insn) || modified_in_p (info.base, insn))
> > +       break;
> > +
> > +      if (forwarding_store_p (insn, info))
> > +       return insn;
> > +    }
> > +
> > +  return NULL;
> > +}
> > +
> > +/* Replace PAIR with two individually-recognized scalar loads.  Load the
> > +   non-base destination first when the low destination overlaps the base.  */
> > +
> > +static bool
> > +split_ldp (rtx_insn *pair, const ldp_info &info)
> > +{
> > +  rtx mems[2] = {
> > +    adjust_address_nv (info.mem, info.mode, 0),
> > +    adjust_address_nv (info.mem, info.mode, info.access_size)
> > +  };
> > +
> > +  int order[2] = { 0, 1 };
> > +  if (reg_overlap_mentioned_p (info.base, info.dests[0]))
> > +    std::swap (order[0], order[1]);
> > +
> > +  start_sequence ();
> > +  emit_insn (gen_rtx_SET (info.dests[order[0]], mems[order[0]]));
> > +  emit_insn (gen_rtx_SET (info.dests[order[1]], mems[order[1]]));
> > +  rtx_insn *seq = get_insns ();
> > +  end_sequence ();
> > +
> > +  unshare_all_rtl_in_chain (seq);
> > +  for (rtx_insn *insn = seq; insn; insn = NEXT_INSN (insn))
> > +    if (!NONDEBUG_INSN_P (insn) || recog_memoized (insn) < 0)
> > +      {
> > +       if (dump_file)
> > +         fprintf (dump_file,
> > +                  "Rejecting LDP %d: scalar replacement is not recognized\n",
> > +                  INSN_UID (pair));
> > +       return false;
> > +      }
> > +
> > +  if (dump_file)
> > +    {
> > +      fprintf (dump_file, "Splitting LDP %d into:\n", INSN_UID (pair));
> > +      for (rtx_insn *insn = seq; insn; insn = NEXT_INSN (insn))
> > +       print_rtl_single (dump_file, insn);
> > +    }
> > +
> > +  emit_insn_before_setloc (seq, pair, INSN_LOCATION (pair));
> > +  delete_insn (pair);
> > +  return true;
> > +}
> > +
> > +const pass_data pass_data_split_ldp_stlf =
> > +{
> > +  RTL_PASS, /* type.  */
> > +  "split_ldp_stlf", /* name.  */
> > +  OPTGROUP_NONE, /* optinfo_flags.  */
> > +  TV_MACH_DEP, /* tv_id.  */
> > +  0, /* properties_required.  */
> > +  0, /* properties_provided.  */
> > +  0, /* properties_destroyed.  */
> > +  0, /* todo_flags_start.  */
> > +  TODO_df_verify | TODO_df_finish, /* todo_flags_finish.  */
> > +};
> > +
> > +class pass_split_ldp_stlf : public rtl_opt_pass
> > +{
> > +public:
> > +  pass_split_ldp_stlf (gcc::context *ctxt)
> > +    : rtl_opt_pass (pass_data_split_ldp_stlf, ctxt)
> > +  {}
> > +
> > +  bool gate (function *fn) final override
> > +  {
> > +    return (optimize >= 1
> > +           && !optimize_debug
> > +           && reload_completed
> > +           && aarch64_split_ldp_stlf
> > +           && !optimize_function_for_size_p (fn)
> > +           && !fn->can_throw_non_call_exceptions);
> > +  }
> > +
> > +  unsigned int execute (function *fn) final override
> > +  {
> > +    unsigned int split_count = 0;
> > +    basic_block bb;
> > +
> > +    FOR_EACH_BB_FN (bb, fn)
> > +      {
> > +       rtx_insn *insn;
> > +       rtx_insn *next;
> > +       FOR_BB_INSNS_SAFE (bb, insn, next)
> > +         {
> > +           ldp_info info;
> > +           if (!extract_ldp (insn, &info))
> > +             continue;
> > +
> > +           rtx_insn *store = find_forwarding_store (insn, info);
> > +           if (!store)
> > +             continue;
> > +
> > +           if (dump_file)
> > +             {
> > +               fprintf (dump_file, "Found forwarding store for LDP %d:\n",
> > +                        INSN_UID (insn));
> > +               print_rtl_single (dump_file, store);
> > +             }
> > +
> > +           split_count += split_ldp (insn, info);
> > +         }
> > +      }
> > +
> > +    if (dump_file)
> > +      fprintf (dump_file, "Split %u LDP instructions\n", split_count);
> > +    return 0;
> > +  }
> > +};
> > +
> > +} // namespace
> > +
> > +rtl_opt_pass *
> > +make_pass_split_ldp_stlf (gcc::context *ctxt)
> > +{
> > +  return new pass_split_ldp_stlf (ctxt);
> > +}
> > diff --git a/gcc/config/aarch64/aarch64-passes.def b/gcc/config/aarch64/aarch64-passes.def
> > index 2df7459af92..594348a0532 100644
> > --- a/gcc/config/aarch64/aarch64-passes.def
> > +++ b/gcc/config/aarch64/aarch64-passes.def
> > @@ -26,4 +26,5 @@ INSERT_PASS_BEFORE (pass_late_thread_prologue_and_epilogue, 1, pass_late_track_s
> >  INSERT_PASS_BEFORE (pass_shorten_branches, 1, pass_insert_bti);
> >  INSERT_PASS_BEFORE (pass_early_remat, 1, pass_ldp_fusion);
> >  INSERT_PASS_BEFORE (pass_peephole2, 1, pass_ldp_fusion);
> > +INSERT_PASS_BEFORE (pass_sched2, 1, pass_split_ldp_stlf);
> >  INSERT_PASS_BEFORE (pass_free_cfg, 1, pass_narrow_gp_writes);
> > diff --git a/gcc/config/aarch64/aarch64-protos.h b/gcc/config/aarch64/aarch64-protos.h
> > index bcc833cfaa1..ec8be386c29 100644
> > --- a/gcc/config/aarch64/aarch64-protos.h
> > +++ b/gcc/config/aarch64/aarch64-protos.h
> > @@ -1268,6 +1268,7 @@ rtl_opt_pass *make_pass_late_track_speculation (gcc::context *);
> >  rtl_opt_pass *make_pass_insert_bti (gcc::context *ctxt);
> >  rtl_opt_pass *make_pass_switch_pstate_sm (gcc::context *ctxt);
> >  rtl_opt_pass *make_pass_ldp_fusion (gcc::context *);
> > +rtl_opt_pass *make_pass_split_ldp_stlf (gcc::context *);
> >  rtl_opt_pass *make_pass_narrow_gp_writes (gcc::context *);
> >
> >  poly_uint64 aarch64_regmode_natural_size (machine_mode);
> > diff --git a/gcc/config/aarch64/aarch64.opt b/gcc/config/aarch64/aarch64.opt
> > index deccc6c8887..21e9500996d 100644
> > --- a/gcc/config/aarch64/aarch64.opt
> > +++ b/gcc/config/aarch64/aarch64.opt
> > @@ -461,6 +461,11 @@ individual writeback accesses where possible.  A value of two means we
> >  also try to opportunistically form writeback opportunities by folding in
> >  trailing destructive updates of the base register used by a pair.
> >
> > +-param=aarch64-split-ldp-stlf=
> > +Target Joined UInteger Var(aarch64_split_ldp_stlf) Init(0) IntegerRange(0, 32) Param
> > +Maximum number of preceding instructions to search when splitting an LDP
> > +to improve store-to-load forwarding.  A value of zero disables the pass.
> > +
> >  -param=aarch64-tag-memory-loop-threshold=
> >  Target Joined UInteger Var(aarch64_tag_memory_loop_threshold) Init(10) IntegerRange(0, 65536) Param
> >  Param to control the threshold in number of granules beyond which an
> > diff --git a/gcc/config/aarch64/t-aarch64 b/gcc/config/aarch64/t-aarch64
> > index 3942a1049b9..936540aeb4e 100644
> > --- a/gcc/config/aarch64/t-aarch64
> > +++ b/gcc/config/aarch64/t-aarch64
> > @@ -238,6 +238,12 @@ aarch64-ldp-fusion.o: $(srcdir)/config/aarch64/aarch64-ldp-fusion.cc \
> >         $(COMPILER) -c $(ALL_COMPILERFLAGS) $(ALL_CPPFLAGS) $(INCLUDES) \
> >                 $(srcdir)/config/aarch64/aarch64-ldp-fusion.cc
> >
> > +aarch64-ldp-split.o: $(srcdir)/config/aarch64/aarch64-ldp-split.cc \
> > +    $(CONFIG_H) $(SYSTEM_H) $(CORETYPES_H) $(BACKEND_H) $(RTL_H) $(DF_H) \
> > +    $(INSN_ATTR_H) insn-config.h $(RECOG_H) tree-pass.h
> > +       $(COMPILER) -c $(ALL_COMPILERFLAGS) $(ALL_CPPFLAGS) $(INCLUDES) \
> > +               $(srcdir)/config/aarch64/aarch64-ldp-split.cc
> > +
> >  aarch64-sched-dispatch.o: $(srcdir)/config/aarch64/aarch64-sched-dispatch.cc \
> >      $(CONFIG_H) $(SYSTEM_H) $(CORETYPES_H) $(BACKEND_H) $(RTL_H) \
> >      $(INSN_ATTR_H) $(REGSET_H) sched-int.h $(DUMPFILE_H) \
> > diff --git a/gcc/doc/params.texi b/gcc/doc/params.texi
> > index 26b34a59aa9..1ae93ae6c75 100644
> > --- a/gcc/doc/params.texi
> > +++ b/gcc/doc/params.texi
> > @@ -2061,6 +2061,14 @@ accesses where possible.  A value of two means we also try to opportunistically
> >  form writeback opportunities by folding in trailing destructive updates of the
> >  base register used by a pair.
> >
> > +@paindex aarch64-split-ldp-stlf
> > +@item aarch64-split-ldp-stlf
> > +Maximum number of preceding instructions that the AArch64 backend searches
> > +for a scalar store that writes one half of a later @code{ldp}.  When such a
> > +store is found, the backend can split the @code{ldp} into two scalar loads to
> > +improve store-to-load forwarding.  A value of zero, which is the default,
> > +disables the transformation.
> > +
> >  @paindex aarch64-loop-vect-issue-rate-niters
> >  @item aarch64-loop-vect-issue-rate-niters
> >  The tuning for some AArch64 CPUs tries to take both latencies and issue
> > diff --git a/gcc/testsuite/gcc.target/aarch64/split-ldp-stlf-1.c b/gcc/testsuite/gcc.target/aarch64/split-ldp-stlf-1.c
> > new file mode 100644
> > index 00000000000..ca4ca4821b1
> > --- /dev/null
> > +++ b/gcc/testsuite/gcc.target/aarch64/split-ldp-stlf-1.c
> > @@ -0,0 +1,43 @@
> > +/* { dg-options "-O2 --param=aarch64-split-ldp-stlf=16" } */
> > +
> > +__attribute__ ((noinline, noclone))
> > +long
> > +split_low (long *p, long value)
> > +{
> > +  p[1] = value;
> > +  __asm__ __volatile__ ("" ::: "memory");
> > +  return p[1] + p[2];
> > +}
> > +
> > +__attribute__ ((noinline, noclone))
> > +long
> > +split_high (long *p, long value)
> > +{
> > +  p[2] = value;
> > +  __asm__ __volatile__ ("" ::: "memory");
> > +  return p[1] + p[2];
> > +}
> > +
> > +__attribute__ ((noinline, noclone))
> > +int
> > +split_low_si (int *p, int value)
> > +{
> > +  p[1] = value;
> > +  __asm__ __volatile__ ("" ::: "memory");
> > +  return p[1] + p[2];
> > +}
> > +
> > +extern void consume (long, long);
> > +
> > +__attribute__ ((noinline, noclone))
> > +void
> > +split_base_dest_overlap (long *p, long value)
> > +{
> > +  p[1] = value;
> > +  __asm__ __volatile__ ("" ::: "memory");
> > +  consume (p[1], p[2]);
> > +}
> > +
> > +/* { dg-final { scan-assembler-not {\tldp\t} } } */
> > +/* { dg-final { scan-assembler-times {\tldr\t} 8 } } */
> > +/* { dg-final { scan-assembler-times {\tldr\tx1, \[x0, 16\]\n\tldr\tx0, \[x0, 8\]} 1 } } */
> > diff --git a/gcc/testsuite/gcc.target/aarch64/split-ldp-stlf-2.c b/gcc/testsuite/gcc.target/aarch64/split-ldp-stlf-2.c
> > new file mode 100644
> > index 00000000000..2ad7e2d5825
> > --- /dev/null
> > +++ b/gcc/testsuite/gcc.target/aarch64/split-ldp-stlf-2.c
> > @@ -0,0 +1,13 @@
> > +/* { dg-options "-O2" } */
> > +
> > +__attribute__ ((noinline, noclone))
> > +long
> > +disabled_by_default (long *p, long value)
> > +{
> > +  p[1] = value;
> > +  __asm__ __volatile__ ("" ::: "memory");
> > +  return p[1] + p[2];
> > +}
> > +
> > +/* { dg-final { scan-assembler-times {\tldp\t} 1 } } */
> > +/* { dg-final { scan-assembler-not {\tldr\t} } } */
> > diff --git a/gcc/testsuite/gcc.target/aarch64/split-ldp-stlf-3.c b/gcc/testsuite/gcc.target/aarch64/split-ldp-stlf-3.c
> > new file mode 100644
> > index 00000000000..930ada6836d
> > --- /dev/null
> > +++ b/gcc/testsuite/gcc.target/aarch64/split-ldp-stlf-3.c
> > @@ -0,0 +1,22 @@
> > +/* { dg-options "-O2 --param=aarch64-split-ldp-stlf=16" } */
> > +
> > +__attribute__ ((noinline, noclone))
> > +long
> > +no_overlap (long *p, long value)
> > +{
> > +  p[4] = value;
> > +  __asm__ __volatile__ ("" ::: "memory");
> > +  return p[1] + p[2];
> > +}
> > +
> > +__attribute__ ((noinline, noclone))
> > +long
> > +wrong_store_width (long *p, int value)
> > +{
> > +  *(int *) ((char *) p + 8) = value;
> > +  __asm__ __volatile__ ("" ::: "memory");
> > +  return p[1] + p[2];
> > +}
> > +
> > +/* { dg-final { scan-assembler-times {\tldp\t} 2 } } */
> > +/* { dg-final { scan-assembler-not {\tldr\t} } } */
> > diff --git a/gcc/testsuite/gcc.target/aarch64/split-ldp-stlf-4.c b/gcc/testsuite/gcc.target/aarch64/split-ldp-stlf-4.c
> > new file mode 100644
> > index 00000000000..3e08bcbea05
> > --- /dev/null
> > +++ b/gcc/testsuite/gcc.target/aarch64/split-ldp-stlf-4.c
> > @@ -0,0 +1,14 @@
> > +/* { dg-options "-O2 --param=aarch64-split-ldp-stlf=16" } */
> > +/* { dg-additional-options "-fnon-call-exceptions" } */
> > +
> > +__attribute__ ((noinline, noclone))
> > +long
> > +preserve_faulting_pair (long *p, long value)
> > +{
> > +  p[1] = value;
> > +  __asm__ __volatile__ ("" ::: "memory");
> > +  return p[1] + p[2];
> > +}
> > +
> > +/* { dg-final { scan-assembler-times {\tldp\t} 1 } } */
> > +/* { dg-final { scan-assembler-not {\tldr\t} } } */
> > diff --git a/gcc/testsuite/gcc.target/aarch64/split-ldp-stlf-5.c b/gcc/testsuite/gcc.target/aarch64/split-ldp-stlf-5.c
> > new file mode 100644
> > index 00000000000..a19551c4374
> > --- /dev/null
> > +++ b/gcc/testsuite/gcc.target/aarch64/split-ldp-stlf-5.c
> > @@ -0,0 +1,13 @@
> > +/* { dg-options "-Os --param=aarch64-split-ldp-stlf=16" } */
> > +
> > +__attribute__ ((noinline, noclone))
> > +long
> > +optimize_for_size (long *p, long value)
> > +{
> > +  p[1] = value;
> > +  __asm__ __volatile__ ("" ::: "memory");
> > +  return p[1] + p[2];
> > +}
> > +
> > +/* { dg-final { scan-assembler-times {\tldp\t} 1 } } */
> > +/* { dg-final { scan-assembler-not {\tldr\t} } } */
> > diff --git a/gcc/testsuite/gcc.target/aarch64/split-ldp-stlf-6.c b/gcc/testsuite/gcc.target/aarch64/split-ldp-stlf-6.c
> > new file mode 100644
> > index 00000000000..bd59fe77b47
> > --- /dev/null
> > +++ b/gcc/testsuite/gcc.target/aarch64/split-ldp-stlf-6.c
> > @@ -0,0 +1,13 @@
> > +/* { dg-options "-O2 --param=aarch64-split-ldp-stlf=1" } */
> > +
> > +__attribute__ ((noinline, noclone))
> > +long
> > +outside_search_window (long *p, long value)
> > +{
> > +  p[1] = value;
> > +  __asm__ __volatile__ ("" ::: "memory");
> > +  return p[1] + p[2];
> > +}
> > +
> > +/* { dg-final { scan-assembler-times {\tldp\t} 1 } } */
> > +/* { dg-final { scan-assembler-not {\tldr\t} } } */
> > --
> > 2.43.0
> >