From: Kyrylo Tkachov <[email protected]>
The documentation of the reduc_widen_ssum and reduc_widen_usum patterns says
only that operand 1 is added to operand 2, which reads as though each element
of operand 1 has to be accumulated into the element of operand 0 at the same
position. Nothing says the patterns are only ever used for a reassociable
reduction, so a port cannot tell whether it is allowed to regroup the input
elements, for example by using a pairwise widening add.
vect_recog_widen_sum_pattern only forms WIDEN_SUM_EXPR
through vect_reassociating_reduction_p, which requires the statement to be a
reduction, refuses a statement nested in the inner loop of an outer-loop
vectorization because the order of the computation matters there, and refuses
a type that needs a fold-left reduction. The comment on the pattern already
describes the idiom as producing N/2 results by summing up pairs of
intermediate results. The accumulator is only ever consumed by a horizontal
sum in the epilogue, so the distribution of input elements over accumulator
elements is not observable.
Spell that out in md.texi, along with the constraint that comes with it: an
implementation that adds elements together before they reach the element width
of operand 0 has to widen them first so that no intermediate sum can overflow.
Note that the widen_[us]sum names are left free for a lane preserving pattern.
Give WIDEN_SUM_EXPR the same statement in tree.def. DOT_PROD_EXPR and SAD_EXPR
are formed through vect_reassociating_reduction_p as well and read the same way,
so add a sentence to each pointing at the WIDEN_SUM_EXPR rule.
Ok for trunk?
Thanks,
Kyrill
gcc/ChangeLog:
* doc/md.texi (reduc_widen_ssum@var{n}@var{m}3)
(reduc_widen_usum@var{n}@var{m}3): Document that the assignment of
input elements to accumulator elements is unconstrained, that
intermediate sums must not overflow, and that the widen_[us]sum
names are reserved. Fix a typo.
* tree.def (WIDEN_SUM_EXPR): Document that the assignment of elements
of the first argument to elements of the second is unconstrained.
(DOT_PROD_EXPR, SAD_EXPR): Note the same freedom.
Signed-off-by: Kyrylo Tkachov <[email protected]>
---
gcc/doc/md.texi | 23 +++++++++++++++++++++--
gcc/tree.def | 19 +++++++++++++++++--
2 files changed, 38 insertions(+), 4 deletions(-)
diff --git a/gcc/doc/md.texi b/gcc/doc/md.texi
index 4b3dc950aeb..f86fa8038e8 100644
--- a/gcc/doc/md.texi
+++ b/gcc/doc/md.texi
@@ -5097,10 +5097,29 @@ in operand 0, which is of the same mode as operand 3.
@itemx @samp{reduc_widen_usum@var{n}@var{m}3}
Operands 0 and 2 are of the same mode, which is wider than the mode of
operand 1. Add operand 1 to operand 2 and place the widened result in
-operand 0. (This is used express accumulation of elements into an accumulator
-of a wider mode.)
+operand 0. (This is used to express accumulation of elements into an
+accumulator of a wider mode.)
@var{m} is the mode of operand 1 and @var{n} is the mode of operand 0.
+These patterns are only used for a reduction whose summation the vectorizer
+has already established may be reassociated. Operand 0 is the accumulator
+of that reduction and is itself reduced to a scalar by a horizontal sum once
+the loop is done, so which element of the accumulator a given input element
+ends up in is not observable. An implementation is therefore free to choose
+which elements of operand 1 it accumulates into which element of operand 0,
+provided that every element of operand 1 is accumulated exactly once. In
+particular it may add adjacent elements of operand 1 to each other before
+accumulating them, which is what a pairwise widening add instruction does.
+
+The intermediate sums must not overflow. An implementation that combines
+elements of operand 1 with each other before they reach the element width of
+operand 0 therefore has to widen them first.
+
+The names @samp{widen_ssum@var{n}@var{m}3} and
+@samp{widen_usum@var{n}@var{m}3} are deliberately left free for a future
+lane preserving pattern, which would have to add element @var{i} of
+operand 1 to element @var{i} of operand 2 for every @var{i}.
+
@mdindex smulhs@var{m}3
@mdindex umulhs@var{m}3
@item @samp{smulhs@var{m}3}
diff --git a/gcc/tree.def b/gcc/tree.def
index ad8e676e942..2e767dbe045 100644
--- a/gcc/tree.def
+++ b/gcc/tree.def
@@ -1465,7 +1465,10 @@ DEFTREECODE (TRANSACTION_EXPR, "transaction_expr", tcc_expression, 1)
arg3 = PLUS_EXPR (tmp, arg3);
or:
tmp = WIDEN_MULT_EXPR(arg1, arg2);
- arg3 = WIDEN_SUM_EXPR (tmp, arg3); */
+ arg3 = WIDEN_SUM_EXPR (tmp, arg3);
+ Like WIDEN_SUM_EXPR, this code is only created for a reduction whose
+ summation may be reassociated, so which products are added into which
+ element of arg3 is not observable. */
DEFTREECODE (DOT_PROD_EXPR, "dot_prod_expr", tcc_expression, 3)
/* Widening summation.
@@ -1474,7 +1477,16 @@ DEFTREECODE (DOT_PROD_EXPR, "dot_prod_expr", tcc_expression, 3)
the size of t1. The type of the entire expression is also t2.
WIDEN_SUM_EXPR is equivalent to first widening (promoting)
the first argument from type t1 to type t2, and then summing it
- with the second argument. */
+ with the second argument.
+
+ This code is only created for a reduction whose summation the vectorizer
+ has established may be reassociated, and the result is reduced to a scalar
+ by a horizontal sum once the loop is done. Which element of the second
+ argument a given element of the first argument is added to is therefore
+ not observable, and an expander may choose that assignment freely, so long
+ as every element of the first argument is accumulated exactly once and no
+ intermediate sum overflows. See the reduc_widen_ssum and reduc_widen_usum
+ patterns in md.texi. */
DEFTREECODE (WIDEN_SUM_EXPR, "widen_sum_expr", tcc_binary, 2)
/* Widening sad (sum of absolute differences).
@@ -1490,6 +1502,9 @@ DEFTREECODE (WIDEN_SUM_EXPR, "widen_sum_expr", tcc_binary, 2)
tmp = IFN_VEC_WIDEN_MINUS_EXPR (arg1, arg2)
tmp2 = ABS_EXPR (tmp)
arg3 = WIDEN_SUM_EXPR (tmp2, arg3)
+ Like WIDEN_SUM_EXPR, this code is only created for a reduction whose
+ summation may be reassociated, so which absolute differences are added
+ into which element of arg3 is not observable.
*/
DEFTREECODE (SAD_EXPR, "sad_expr", tcc_expression, 3)
--
2.50.1 (Apple Git-155)
lmpx.com only provides a reader for public news (NNTP) servers. It is not
affiliated with the servers or forums shown here and is not responsible for
the content of articles, which is written by their respective authors.