[PATCH v2 00/13] vect: Add support for HSSR (formerly FFR)
Alfie Richards <[email protected]>
| Newsgroups | gmane.comp.gcc.patches |
|---|---|
| Message-ID | <[email protected]> |
Hi All,
Heres my V2 for this patch series after some more time in the oven.
I'm hoping this is now in a state for a more in depth review.
Major changes:
--------------
It's now called "Hardware Safe Speculative Reads". This is mostly because FFR
was a bad name, as it is aarch64 specific and doesn't describe what the
feature actually does. I'm not attached to the new name if anyone fancies bike
shedding.
As was discussed in the previous review, this now advances the IV's by the
number of loaded iterations instead of repeating iterations to prese alignment.
I've also added support for versioning loops to choose between mutually aligned
loops and HSSR loops. This patch is the hackiest part of the series and I would
appreciate any feedback and ideas for better ways to do this.
I added some basic HSSR costing, but there is definitely room for improvement.
Otherwise, much of the rest of the design of the V1 remains.
The series depends on Tamars early break patch series for some minor optabs and
internal functions (I think only COUNT_ACTIVE).
Known problems/limitations:
---------------------------
There are some hefty areas for improvement with this patch series. I think these
should all now be addressed in later patch series as this series is already
large. However, this may be useful context.
Firstly, ivopts and addressing is a bit of a mess. It seems ivopts cannot
handle the iv step being loop variant. So the IV's in generated code
are a mess and the addressing for the loads are sub-optimal.
Secondly, the HSSR loop versioning creates its own scalar loop rather than
re-using the scalar loop any versioning for the main loop. Safety checks may
also be duplicated.
ie, current HSSR versioning looks roughtly like:
│
▼
┌──non-alignment versioning────────────┐
│ checks │
┌──────▼──────┐ ▼
│ Scalar Loop │ ┌──────alignment checks─────┐
└──────┬──────┘ ▼ ┌──────────▼────────────┐
│ ┌───hssr safety checks───┐ │ Mutually aligned loop │
│ ┌──────▼──────┐ ┌──────▼────┐ └───────────┬───────────┘
│ │ Scalar Loop │ │ HSSR Loop │ │
│ └──────┬──────┘ └──────┬────┘ │
│ │ │ │
│ │ ┌─────▼────┐ ┌─────▼────┐
│ │ │ epilogue │ │ epilogue │
│ │ └─────┬────┘ └─────┬────┘
└─────────┴──┬─────────────────────┴──────────────────┘
▼
There is obviously some room for code size improvement here. Though it is
complicated by the fact the HSSR loop and mutually aligned loop can use
different vectorization modes so sharing safety checks and epilogues is likely
not possible.
Thirdly, I haven't implemented any support for HSSR gather/strided loads. This
limits the situations this can be used. I plan to add this later.
Additionally, I think it would be possible to add some limited multi-lane slp
support (for aarch64 at least) which this doesn't. Such support would be quite
complicated and limited.
Lastly, there is no support for HSSR in other architectures. As pointed out
previously, other architectures could add support for HSSR. I have tried to
design support to make that as easy as possible, but haven't experimented with
adding that support.
Testing:
--------
Bootstrapped, O3+LTO bootstrapped, and regression tested on aarch64, x86_64, and
armhf. Spec v6 and v8 built and run successfully with this series for aarch64.
Thoughts and feedback? Okay for trunk? :)
Many thanks,
Alfie
Alfie Richards (13):
vect: Change IV increment position to the end of the loop
aarch64: Make reads/writes to FFR expensive
aarch64: Add fold_contiguous_load helper function.
ifn: optab: aarch64: Add optabs and internal functions for hssr
operations.
vect: Add SLP_NODE argument to vect_get_loop_mask.
vect: Make vect_maybe_permute_loop_masks optional and retargetable
vect: Add EXCLUDE_VIRTUALS argument to _slp_tree::push_vec_def.
vect: Add vect-hssr-usage param.
vect: Add hssr_read value to vect_cost_for_stmt
vect: Add HSSR analysis
vect: Add HSSR transformation.
vect: Add HSSR versioning.
vect: Enable HSSR by default.
gcc/config/aarch64/aarch64-acle-builtins.cc | 24 +
gcc/config/aarch64/aarch64-acle-builtins.h | 1 +
gcc/config/aarch64/aarch64-protos.h | 10 +-
.../aarch64/aarch64-sve-builtins-base.cc | 94 ++--
gcc/config/aarch64/aarch64-sve.md | 222 +++++++-
gcc/config/aarch64/aarch64.cc | 36 +-
gcc/config/aarch64/tuning_models/a64fx.h | 3 +-
gcc/config/aarch64/tuning_models/cortexa320.h | 3 +-
gcc/config/aarch64/tuning_models/cortexx925.h | 3 +-
gcc/config/aarch64/tuning_models/generic.h | 3 +-
.../aarch64/tuning_models/generic_armv8_a.h | 3 +-
.../aarch64/tuning_models/generic_armv9_a.h | 3 +-
gcc/config/aarch64/tuning_models/hip12.h | 3 +-
.../aarch64/tuning_models/neoverse512tvb.h | 3 +-
gcc/config/aarch64/tuning_models/neoversen2.h | 3 +-
gcc/config/aarch64/tuning_models/neoversen3.h | 3 +-
gcc/config/aarch64/tuning_models/neoversev1.h | 3 +-
gcc/config/aarch64/tuning_models/neoversev2.h | 3 +-
gcc/config/aarch64/tuning_models/neoversev3.h | 3 +-
.../aarch64/tuning_models/neoversev3ae.h | 3 +-
gcc/config/aarch64/tuning_models/olympus.h | 3 +-
gcc/internal-fn.cc | 25 +
gcc/internal-fn.def | 18 +
gcc/optabs-tree.cc | 57 ++
gcc/optabs-tree.h | 2 +
gcc/optabs.def | 8 +
gcc/params.opt | 4 +
gcc/target.h | 7 +-
gcc/targhooks.cc | 3 +
gcc/testsuite/gcc.target/aarch64/sve/ffr_1.c | 16 +
gcc/testsuite/gcc.target/aarch64/sve/ffr_11.c | 13 +
gcc/testsuite/gcc.target/aarch64/sve/ffr_12.c | 17 +
gcc/testsuite/gcc.target/aarch64/sve/ffr_13.c | 15 +
gcc/testsuite/gcc.target/aarch64/sve/ffr_14.c | 18 +
gcc/testsuite/gcc.target/aarch64/sve/ffr_2.c | 35 ++
gcc/testsuite/gcc.target/aarch64/sve/ffr_3.c | 44 ++
gcc/testsuite/gcc.target/aarch64/sve/ffr_4.c | 25 +
gcc/testsuite/gcc.target/aarch64/sve/ffr_5.c | 25 +
gcc/testsuite/gcc.target/aarch64/sve/ffr_6.c | 43 ++
.../gcc.target/aarch64/sve/ffr_6_run.c | 82 +++
gcc/testsuite/gcc.target/aarch64/sve/ffr_7.c | 19 +
gcc/testsuite/gcc.target/aarch64/sve/ffr_8.c | 17 +
gcc/testsuite/gcc.target/aarch64/sve/ffr_9.c | 18 +
.../gcc.target/aarch64/sve/noeffect11.c | 2 +-
.../gcc.target/aarch64/sve/peel_ind_1.c | 2 +-
.../gcc.target/aarch64/sve/peel_ind_12.c | 2 +-
.../gcc.target/aarch64/sve/peel_ind_13.c | 2 +-
.../gcc.target/aarch64/sve/peel_ind_14.c | 2 +-
.../gcc.target/aarch64/sve/peel_ind_15.c | 2 +-
.../gcc.target/aarch64/sve/peel_ind_16.c | 2 +-
.../gcc.target/aarch64/sve/peel_ind_17.c | 2 +-
.../gcc.target/aarch64/sve/peel_ind_2.c | 2 +-
.../gcc.target/aarch64/sve/peel_ind_3.c | 2 +-
.../gcc.target/aarch64/sve/peel_ind_4.c | 2 +-
.../gcc.target/aarch64/sve/peel_ind_5.c | 2 +-
.../gcc.target/aarch64/sve/peel_ind_6.c | 2 +-
.../gcc.target/aarch64/sve/peel_ind_7.c | 2 +-
.../gcc.target/aarch64/sve/peel_ind_8.c | 2 +-
.../gcc.target/aarch64/sve/peel_ind_9.c | 2 +-
gcc/testsuite/gcc.target/aarch64/sve/peeled.c | 2 +-
.../gcc.target/aarch64/sve/peeled1.c | 2 +-
.../gcc.target/aarch64/sve/peeled2.c | 2 +-
.../gcc.target/aarch64/sve/peeled3.c | 2 +-
.../gcc.target/aarch64/sve/pfalse-load.c | 7 +-
gcc/tree-data-ref.cc | 2 +
gcc/tree-ssa-alias.cc | 1 +
gcc/tree-ssa-loop-ivopts.cc | 2 +
gcc/tree-vect-data-refs.cc | 3 +-
gcc/tree-vect-loop-manip.cc | 515 +++++++++++++++++-
gcc/tree-vect-loop.cc | 453 ++++++++++++++-
gcc/tree-vect-slp.cc | 36 +-
gcc/tree-vect-stmts.cc | 166 ++++--
gcc/tree-vectorizer.cc | 50 +-
gcc/tree-vectorizer.h | 103 +++-
74 files changed, 2115 insertions(+), 206 deletions(-)
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/ffr_1.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/ffr_11.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/ffr_12.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/ffr_13.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/ffr_14.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/ffr_2.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/ffr_3.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/ffr_4.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/ffr_5.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/ffr_6.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/ffr_6_run.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/ffr_7.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/ffr_8.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/ffr_9.c
--
2.43.0