[PATCH 2/3] ipa: New indirect info kind and optimization for PMF
Chi Wang <[email protected]>
| Newsgroups | gmane.comp.gcc.patches |
|---|---|
| Message-ID | <[email protected]> |
This patch continues the IFN_PTRMEMFUNC_OBJ optimizations and implements IPA related optimizations. A new indirect info cgraph_ptrmemfunc_indirect_info is added. The indirect info tracks recognized PMF calls. Currently there are two paths to the creation: 1. Through the existing PMF call discovery, and 2. Through the early detection of IFN_PTRMEMFUNC_OBJ. The second case is necessary for LTO to retain related info in memory, which will be addressed in the next patch. The motivation behind adding a new indirect info type is that PMF call needs more type information and also two aggregate offsets to recover pfn and delta. Adding the relevant optional field bloats the indirect call by a lot, considering PMF is rare and indirect call is far more common. In addition, it is easier to dispatch related optimization path simply based on the indirect info types. The optimization basically is a mix between GIMPLE pass in the first patch and the IPA try_make_edge_direct_* optimizations. The edge is first distinguished by the type of the pfn field. If it is an address to a function declaration, it is a simple indirect call case and it follows the previous optimizations. If it is an integer constant, both pfn and delta are inspected to check whether it is a polymorphic call. The pass then tries to query if it is already a direct target, and resolves using IPA devirtualization if it is not. It applies to both IPA inlining and IPA-CP. This optimization should let IPA be able to identify inlining and cloning opportunities that were previously impossible, which is common when using std::invoke on PMFs. This patch also removes cgraph_simple_indirect_info::member_ptr as it is no longer in use. gcc/ChangeLog: * cgraph.cc (cgraph_node::create_indirect_edge): Identify PMF calls early and create cgraph_ptrmemfunc_indirect_info. (cgraph_indirect_call_info::dump): Remove member_ptr output and add PMF specific dump output. * cgraph.h (enum cgraph_indirect_info_kind): Add CIIK_PTRMEMFUNC. (cgraph_indirect_call_info): Widen kind. (cgraph_ptrmemfunc_indirect_info): New class. (cgraph_simple_indirect_info): Remove member_ptr. * cgraphclones.cc (cgraph_edge::clone): Add cloning for cgraph_ptrmemfunc_indirect_info. * ipa-cp.cc (ipa_get_indirect_edge_target_1): Add direct target inference for cgraph_ptrmemfunc_indirect_info. (ipcp_discover_new_direct_edges): Add direct target discovery for cgraph_ptrmemfunc_indirect_info. * ipa-devirt.cc (ipa_devirt): Update speculative call handling for cgraph_ptrmemfunc_indirect_info. * ipa-prop.cc (ipa_analyze_indirect_call_uses): Change PMF specific information to cgraph_ptrmemfunc_indirect_info. (ipa_make_edge_direct_to_target): Remove sii->member_ptr condition and comment. (try_make_edge_direct_simple_call): Remove reference to PMF in the comments. (ipa_get_ptrmemfunc_direct_target): New function. Compute the direct target of a PMF call. (try_make_edge_direct_ptrmemfunc_call): New function. Recover pfn and delta from PMF to find the direct target. (update_indirect_edges_after_inlining): Dispatch for PMF indirect edge info. (ipa_write_indirect_edge_info): Streaming for cgraph_ptrmemfunc_indirect_info. (ipa_read_indirect_edge_info): Streaming for cgraph_ptrmemfunc_indirect_info. * ipa-prop.h (ipa_get_ptrmemfunc_direct_target): New function. * lto-cgraph.cc (input_edge): LTO streaming for cgraph_ptrmemfunc_indirect_info. gcc/testsuite/ChangeLog: * g++.dg/opt/devirtpmf6.C: New test. * g++.dg/opt/devirtpmf7.C: New test. * g++.dg/opt/devirtpmf8.C: New test. Signed-off-by: Chi Wang <[email protected]> --- gcc/cgraph.cc | 73 ++++++- gcc/cgraph.h | 71 ++++++- gcc/cgraphclones.cc | 5 + gcc/ipa-cp.cc | 26 ++- gcc/ipa-devirt.cc | 10 + gcc/ipa-prop.cc | 279 +++++++++++++++++++++++--- gcc/ipa-prop.h | 3 +- gcc/lto-cgraph.cc | 4 + gcc/testsuite/g++.dg/opt/devirtpmf6.C | 62 ++++++ gcc/testsuite/g++.dg/opt/devirtpmf7.C | 23 +++ gcc/testsuite/g++.dg/opt/devirtpmf8.C | 28 +++ 11 files changed, 540 insertions(+), 44 deletions(-) create mode 100644 gcc/testsuite/g++.dg/opt/devirtpmf6.C create mode 100644 gcc/testsuite/g++.dg/opt/devirtpmf7.C create mode 100644 gcc/testsuite/g++.dg/opt/devirtpmf8.C diff --git a/gcc/cgraph.cc b/gcc/cgraph.cc index 4b22cad9f11..327fbe57563 100644 --- a/gcc/cgraph.cc +++ b/gcc/cgraph.cc @@ -70,6 +70,7 @@ along with GCC; see the file COPYING3. If not see #include "symtab-thunks.h" #include "symtab-clones.h" #include "attr-callback.h" +#include "gimple-ptrmemfunc.h" /* FIXME: Only for PROP_loops, but cgraph shouldn't have to know about this. */ #include "tree-pass.h" @@ -1193,9 +1194,49 @@ cgraph_node::create_indirect_edge (gcall *call_stmt, int ecf_flags, type)); } else if (target && TREE_CODE (target) == SSA_NAME) - edge->indirect_info - = (new (ggc_alloc<cgraph_simple_indirect_info> ()) - cgraph_simple_indirect_info (ecf_flags)); + { + /* Recover calling type info earlier for pointer-to-member function + calls so we can retain vtable definition during WPA. */ + cgraph_ptrmemfunc_indirect_info *pmf = NULL; + gimple *def = SSA_NAME_DEF_STMT (target); + if (gimple_code (def) == GIMPLE_PHI + && gimple_phi_num_args (def) == 2 + && POINTER_TYPE_P (TREE_TYPE (target)) + && TREE_CODE (TREE_TYPE (TREE_TYPE (target))) == METHOD_TYPE) + { + for (int i = 0; i < 2; i++) + { + tree pmf_obj; + if (!gimple_analyze_virtual_ptrmemfunc_branch + (PHI_ARG_DEF (def, i), pmf_obj)) + continue; + gimple *pmf_obj_def = SSA_NAME_DEF_STMT (pmf_obj); + gcc_checking_assert + (gimple_call_internal_p (pmf_obj_def, IFN_PTRMEMFUNC_OBJ)); + tree method_type_token = gimple_call_arg (pmf_obj_def, 1); + tree outer_type_token = gimple_call_arg (pmf_obj_def, 2); + + pmf = (new (ggc_alloc<cgraph_ptrmemfunc_indirect_info> ()) + cgraph_ptrmemfunc_indirect_info (ecf_flags)); + pmf->method_type = TREE_TYPE (TREE_TYPE (method_type_token)); + pmf->base_via_virtual = tree_to_uhwi (method_type_token); + if (tree_fits_shwi_p (outer_type_token)) + { + pmf->outer_type + = TREE_TYPE (TREE_TYPE (outer_type_token)); + pmf->base_offset = tree_to_shwi (outer_type_token); + } + break; + } + } + + if (pmf) + edge->indirect_info = pmf; + else + edge->indirect_info + = (new (ggc_alloc<cgraph_simple_indirect_info> ()) + cgraph_simple_indirect_info (ecf_flags)); + } else edge->indirect_info = (new (ggc_alloc<cgraph_indirect_call_info> ()) @@ -2747,9 +2788,8 @@ cgraph_indirect_call_info::dump (FILE *f, bool newline) const = dyn_cast <const cgraph_simple_indirect_info *> (this)) { if (sii->agg_contents) - fprintf (f, " indirect %s callsite, calling param %i, " + fprintf (f, " indirect aggregate callsite, calling param %i, " "offset " HOST_WIDE_INT_PRINT_DEC ", %s", - sii->member_ptr ? "member ptr" : "aggregate", sii->param_index, sii->offset, sii->by_ref ? "by reference" : "by_value"); else if (sii->param_index >= 0) @@ -2759,6 +2799,29 @@ cgraph_indirect_call_info::dump (FILE *f, bool newline) const fprintf (f, " indirect simple callsite, not calling a known " "parameter"); } + else if (const cgraph_ptrmemfunc_indirect_info *pmf + = dyn_cast <const cgraph_ptrmemfunc_indirect_info *> (this)) + { + fprintf (f, " indirect member ptr callsite, " + "calling param %i, pfn_offset " HOST_WIDE_INT_PRINT_DEC ", " + "delta_offset " HOST_WIDE_INT_PRINT_DEC ", %s", + pmf->param_index, pmf->pfn_offset, pmf->delta_offset, + pmf->by_ref ? "by reference" : "by_value"); + if (pmf->method_type) + { + fprintf (f, ", method_type "); + print_generic_expr (f, pmf->method_type); + } + if (pmf->outer_type) + { + fprintf (f, ", outer_type "); + print_generic_expr (f, pmf->outer_type); + fprintf (f, ", base_offset " HOST_WIDE_INT_PRINT_DEC + ", %s through virtual base", + pmf->base_offset, + pmf->base_via_virtual ? "is" : "not"); + } + } else fprintf (f, " indirect callsite"); diff --git a/gcc/cgraph.h b/gcc/cgraph.h index e4a0a6b1bc7..0724c28c3ef 100644 --- a/gcc/cgraph.h +++ b/gcc/cgraph.h @@ -1734,6 +1734,8 @@ enum cgraph_indirect_info_kind { /* Call of a virtual method when the target is an OBJ_TYPE_REF which conforms to virtual_method_call_p. */ CIIK_POLYMORPHIC, + /* An indirect call through pointer to member function. */ + CIIK_PTRMEMFUNC, /* Must be last */ CIIK_N_KINDS }; @@ -1765,13 +1767,13 @@ public: int param_index; /* Identifier of the specific type of indirect info this actually is. */ - enum cgraph_indirect_info_kind kind : 2; + enum cgraph_indirect_info_kind kind : 3; /* Number of speculative call targets. */ unsigned num_speculative_call_targets : 16; }; /* Structure containing additional information about non-virtual indirect calls - where the target is an SSA_NAME. */ + where the target is an SSA_NAME and not known to be a PMF call. */ class GTY((tag ("CIIK_SIMPLE"))) cgraph_simple_indirect_info : public cgraph_indirect_call_info @@ -1780,7 +1782,7 @@ public: cgraph_simple_indirect_info (int flags) : cgraph_indirect_call_info (CIIK_SIMPLE, flags), offset (0), rec_type (NULL_TREE), fld_offset (0), agg_contents (false), - member_ptr (false), fnptr_loaded_from_record (false), by_ref (false), + fnptr_loaded_from_record (false), by_ref (false), guaranteed_unmodified (false) {} @@ -1798,8 +1800,6 @@ public: /* Set when the call is a call of a pointer loaded from contents of an aggregate at offset. */ unsigned agg_contents : 1; - /* Set when this is a call through a member pointer. */ - unsigned member_ptr : 1; /* Set if the function is a call of a pointer loaded from a record type stored in otr_type at offset offset. */ unsigned fnptr_loaded_from_record : 1; @@ -1864,6 +1864,45 @@ public: unsigned vptr_changed : 1; }; +/* Structure containing additional information about pointer-to-member function + calls. */ + +class GTY ((tag ("CIIK_PTRMEMFUNC"))) + cgraph_ptrmemfunc_indirect_info : public cgraph_indirect_call_info +{ +public: + cgraph_ptrmemfunc_indirect_info (int flags) + : cgraph_indirect_call_info (CIIK_PTRMEMFUNC, flags), by_ref (false), + base_via_virtual (false), pfn_offset (0), delta_offset (0), + method_type (NULL), outer_type (NULL), base_offset (0) + {} + + cgraph_ptrmemfunc_indirect_info (cgraph_simple_indirect_info sii) + : cgraph_indirect_call_info (CIIK_PTRMEMFUNC, sii.ecf_flags), + by_ref (false), base_via_virtual (false), pfn_offset (0), + delta_offset (0), method_type (NULL), outer_type (NULL), base_offset (0) + { + num_speculative_call_targets = sii.num_speculative_call_targets; + } + + /* Whether the destination is loaded from a parameter passed by reference. */ + unsigned by_ref : 1; + /* Whether the base_offset is computed through some virtual base. */ + unsigned base_via_virtual : 1; + + /* Offset of pfn within the aggregate. */ + HOST_WIDE_INT pfn_offset; + /* Offset of delta within the aggregate. */ + HOST_WIDE_INT delta_offset; + /* Method type of the calling pointer-to-member function. */ + tree method_type; + /* Outer type (type of the original C++ FE object) of the called object. */ + tree outer_type; + /* If outer type is not NULL, it is the offset from converting the outer type + to base object type of the function. */ + HOST_WIDE_INT base_offset; +}; + /* Return true if ii is a cgraph_polymorphic_indirect_info that is usable_p. */ inline bool @@ -2491,6 +2530,28 @@ is_a_helper <const cgraph_polymorphic_indirect_info *> return p && p->kind == CIIK_POLYMORPHIC; } +/* Report whether or not THIS indirect info is a known PMF one. */ + +template <> +template <> +inline bool +is_a_helper <cgraph_ptrmemfunc_indirect_info *> +::test (cgraph_indirect_call_info *p) +{ + return p && p->kind == CIIK_PTRMEMFUNC; +} + +/* Likewise, but const qualified. */ + +template <> +template <> +inline bool +is_a_helper <const cgraph_ptrmemfunc_indirect_info *> +::test (const cgraph_indirect_call_info *p) +{ + return p && p->kind == CIIK_PTRMEMFUNC; +} + typedef void (*cgraph_edge_hook)(cgraph_edge *, void *); typedef void (*cgraph_node_hook)(cgraph_node *, void *); typedef void (*varpool_node_hook)(varpool_node *, void *); diff --git a/gcc/cgraphclones.cc b/gcc/cgraphclones.cc index 180ef4e52c8..ff5d55fadb0 100644 --- a/gcc/cgraphclones.cc +++ b/gcc/cgraphclones.cc @@ -132,6 +132,11 @@ cgraph_edge::clone (cgraph_node *n, gcall *call_stmt, unsigned stmt_uid, = (new (ggc_alloc<cgraph_simple_indirect_info> ()) cgraph_simple_indirect_info ( *(const cgraph_simple_indirect_info *) indirect_info)); + else if (indirect_info->kind == CIIK_PTRMEMFUNC) + new_edge->indirect_info + = (new (ggc_alloc<cgraph_ptrmemfunc_indirect_info> ()) + cgraph_ptrmemfunc_indirect_info ( + *(const cgraph_ptrmemfunc_indirect_info *) indirect_info)); else new_edge->indirect_info = (new (ggc_alloc<cgraph_indirect_call_info> ()) diff --git a/gcc/ipa-cp.cc b/gcc/ipa-cp.cc index 431da498921..f35be024ead 100644 --- a/gcc/ipa-cp.cc +++ b/gcc/ipa-cp.cc @@ -3193,6 +3193,28 @@ ipa_get_indirect_edge_target_1 (struct cgraph_edge *ie, else return NULL_TREE; } + if (cgraph_ptrmemfunc_indirect_info *pmf + = dyn_cast <cgraph_ptrmemfunc_indirect_info *> (ie->indirect_info)) + { + tree pfn = NULL_TREE, delta = NULL_TREE; + + auto recover = [&] (HOST_WIDE_INT offset, tree &t) + { + if ((unsigned) param_index < known_csts.length () + && known_csts[param_index]) + t = ipa_find_agg_cst_from_init (known_csts[param_index], offset, + pmf->by_ref); + + if (!t) + t = avs.get_value (param_index, offset / BITS_PER_UNIT, pmf->by_ref); + }; + recover (pmf->pfn_offset, pfn); + if (!pfn) + return NULL_TREE; + recover (pmf->delta_offset, delta); + + return ipa_get_ptrmemfunc_direct_target (ie, pfn, delta); + } if (!opt_for_fn (ie->caller->decl, flag_devirtualize)) return NULL_TREE; @@ -4149,7 +4171,9 @@ ipcp_discover_new_direct_edges (struct cgraph_node *node, = dyn_cast <cgraph_polymorphic_indirect_info *> (ie->indirect_info); cgraph_simple_indirect_info *sii = dyn_cast <cgraph_simple_indirect_info *> (ie->indirect_info); - bool agg_contents = sii && sii->agg_contents; + cgraph_ptrmemfunc_indirect_info *pmf + = dyn_cast <cgraph_ptrmemfunc_indirect_info *> (ie->indirect_info); + bool agg_contents = (sii && sii->agg_contents) || pmf; bool polymorphic = !!pii; int param_index = ie->indirect_info->param_index; struct cgraph_edge *cs = ipa_make_edge_direct_to_target (ie, target, diff --git a/gcc/ipa-devirt.cc b/gcc/ipa-devirt.cc index f70ed21652a..f899959bbd3 100644 --- a/gcc/ipa-devirt.cc +++ b/gcc/ipa-devirt.cc @@ -4036,6 +4036,16 @@ ipa_devirt (void) e->get_next_speculative_id ()); } } + else if (is_a <cgraph_ptrmemfunc_indirect_info *> (e->indirect_info)) + { + if (e->speculative) + { + if (dump_file) + fprintf (dump_file, "Call is already speculated\n\n"); + stats.nspeculated++; + } + continue; + } if (update) ipa_update_overall_fn_summary (n); } diff --git a/gcc/ipa-prop.cc b/gcc/ipa-prop.cc index b0b078b9cd2..3ff51cf5eb8 100644 --- a/gcc/ipa-prop.cc +++ b/gcc/ipa-prop.cc @@ -63,6 +63,7 @@ along with GCC; see the file COPYING3. If not see #include "lto-streamer.h" #include "attribs.h" #include "attr-callback.h" +#include "gimple-ptrmemfunc.h" /* Function summary where the parameter infos are actually stored. */ ipa_node_params_t *ipa_node_params_sum = NULL; @@ -2967,7 +2968,7 @@ ipa_analyze_indirect_call_uses (struct ipa_func_body_info *fbi, gcall *call, cgraph_simple_indirect_info *sii = as_a <cgraph_simple_indirect_info *> (cs->indirect_info); sii->param_index = index; - gcc_assert (!sii->agg_contents && !sii->member_ptr); + gcc_assert (!sii->agg_contents); ipa_set_param_used_by_indirect_call (info, index, true); } return; @@ -3125,14 +3126,32 @@ ipa_analyze_indirect_call_uses (struct ipa_func_body_info *fbi, gcall *call, } cgraph_edge *cs = fbi->node->get_edge (call); - cgraph_simple_indirect_info *sii = - as_a <cgraph_simple_indirect_info *> (cs->indirect_info); - sii->param_index = index; - sii->offset = offset; - sii->agg_contents = 1; - sii->member_ptr = 1; - sii->by_ref = by_ref; - sii->guaranteed_unmodified = 1; + + /* Replace simple indirect info into the PMF specific one. */ + cgraph_ptrmemfunc_indirect_info *pmf; + if (cgraph_simple_indirect_info *sii + = dyn_cast <cgraph_simple_indirect_info *> (cs->indirect_info)) + { + pmf = new (ggc_alloc<cgraph_ptrmemfunc_indirect_info> ()) + cgraph_ptrmemfunc_indirect_info (*sii); + ggc_free (sii); + cs->indirect_info = pmf; + } + else + pmf = as_a <cgraph_ptrmemfunc_indirect_info *> (cs->indirect_info); + + pmf->param_index = index; + pmf->by_ref = by_ref; + + /* Recover both pfn and delta of the PMF aggregate. */ + tree pmf_type = TREE_CODE (rec) == SSA_NAME ? TREE_TYPE (TREE_TYPE (rec)) + : TREE_TYPE (rec); + tree pfn_field, delta_field; + bool is_pmf = type_like_member_ptr_p (pmf_type, &pfn_field, &delta_field); + gcc_checking_assert (is_pmf); + pmf->pfn_offset = int_bit_position (pfn_field); + pmf->delta_offset = int_bit_position (delta_field); + ipa_set_param_used_by_indirect_call (info, index, true); return; } @@ -4005,18 +4024,14 @@ ipa_make_edge_direct_to_target (struct cgraph_edge *ie, tree target, target = canonicalize_constructor_val (target, NULL); if (!target || TREE_CODE (target) != FUNCTION_DECL) { - cgraph_simple_indirect_info *sii - = dyn_cast <cgraph_simple_indirect_info *> (ie->indirect_info); - /* Member pointer call that goes through a VMT lookup. */ - if ((sii && sii->member_ptr) - /* Or if target is not an invariant expression and we do not - know if it will evaluate to function at runtime. - This can happen when folding through &VAR, where &VAR - is IP invariant, but VAR itself is not. - - TODO: It seems that we may try to fold the expression and see - if VAR is readonly. */ - || !is_gimple_ip_invariant (target)) + /* If target is not an invariant expression and we do not + know if it will evaluate to function at runtime. + This can happen when folding through &VAR, where &VAR + is IP invariant, but VAR itself is not. + + TODO: It seems that we may try to fold the expression and see + if VAR is readonly. */ + if (!is_gimple_ip_invariant (target)) { if (dump_enabled_p ()) { @@ -4353,12 +4368,11 @@ try_decrement_rdesc_refcount (struct ipa_jump_func *jfunc) } /* Try to find a destination for indirect edge IE that corresponds to a simple - call or a call of a member function pointer and where the destination is a - pointer formal parameter described by jump function JFUNC. TARGET_TYPE is - the type of the parameter to which the result of JFUNC is passed. If it can - be determined, return the newly direct edge, otherwise return NULL. - NEW_ROOT and NEW_ROOT_INFO is the node and its info that JFUNC lattices are - relative to. */ + call where the destination is a pointer formal parameter described by jump + function JFUNC. TARGET_TYPE is the type of the parameter to which the result + of JFUNC is passed. If it can be determined, return the newly direct edge, + otherwise return NULL. NEW_ROOT and NEW_ROOT_INFO is the node and its info + that JFUNC lattices are relative to. */ static struct cgraph_edge * try_make_edge_direct_simple_call (struct cgraph_edge *ie, @@ -4401,6 +4415,162 @@ try_make_edge_direct_simple_call (struct cgraph_edge *ie, return cs; } +/* Try to find a destination for indirect edge IE that corresponds to a call of + a member function pointer and where the pfn is a known constant PFN_TREE. + DELTA_TREE might be the constant if it is, or NULL it is not known. If it + can be determined, return the new direct target of the call, otherwise return + NULL. */ +tree +ipa_get_ptrmemfunc_direct_target (struct cgraph_edge *ie, tree pfn_tree, + tree delta_tree) +{ + cgraph_ptrmemfunc_indirect_info *pmf + = as_a<cgraph_ptrmemfunc_indirect_info *> (ie->indirect_info); + + /* Non-virtual: if PFN refers to a function. */ + if (TREE_CODE (pfn_tree) == ADDR_EXPR) + { + if (TREE_CODE (TREE_OPERAND (pfn_tree, 0)) == FUNCTION_DECL) + return TREE_OPERAND (pfn_tree, 0); + return NULL; + } + + /* Virtual: if PFN is just an integer constant. */ + if (TREE_CODE (pfn_tree) != INTEGER_CST + || !opt_for_fn (ie->caller->decl, flag_devirtualize) + || !pmf->method_type) + return NULL; + + tree base = TYPE_METHOD_BASETYPE (pmf->method_type); + if (in_lto_p) + base = prevailing_odr_type (base); + + unsigned HOST_WIDE_INT vt_entry_size; + { + if (!TYPE_BINFO (base) || !BINFO_VTABLE (TYPE_BINFO (base))) + return NULL; + tree vt = BINFO_VTABLE (TYPE_BINFO (base)); + if (!vt) + return NULL; + vt_entry_size = tree_to_uhwi (TYPE_SIZE_UNIT (TREE_TYPE (TREE_TYPE (vt)))); + if (vt_entry_size == 0) + return NULL; + } + + /* Decode the offset and delta according to the target. */ + if (!tree_fits_shwi_p (pfn_tree) || !tree_fits_shwi_p (delta_tree)) + return NULL; + HOST_WIDE_INT pfn = tree_to_shwi (pfn_tree); + HOST_WIDE_INT delta = tree_to_shwi (delta_tree); + switch (TARGET_PTRMEMFUNC_VBIT_LOCATION) + { + case ptrmemfunc_vbit_in_pfn: + if (!(pfn & 1)) + return NULL; + pfn -= 1; + break; + case ptrmemfunc_vbit_in_delta: + if (!(delta & 1)) + return NULL; + delta >>= 1; + break; + default: + gcc_unreachable (); + } + + if (pfn % vt_entry_size != 0) + return ipa_impossible_devirt_target (ie, NULL_TREE); + + HOST_WIDE_INT idx = pfn / vt_entry_size; + HOST_WIDE_INT outer_offset = (delta + pmf->base_offset); + HOST_WIDE_INT base_offset = delta; + + tree inner = NULL; + if (pmf->outer_type) + inner = find_outermost_base_at_offset (pmf->outer_type, outer_offset); + else + inner = find_outermost_base_at_offset (base, base_offset); + + if (!inner) + return NULL; + + /* Look up whether the target is final or unreachable. */ + tree fndecl = NULL; + if (pmf->outer_type) + fndecl + = get_ptrmemfunc_polymorphic_direct_target (pmf->outer_type, inner, + outer_offset * BITS_PER_UNIT, + idx); + else + fndecl + = get_ptrmemfunc_polymorphic_direct_target (base, inner, + base_offset * BITS_PER_UNIT, + idx); + + if (fndecl + && fndecl_built_in_p (fndecl, BUILT_IN_UNREACHABLE, + BUILT_IN_UNREACHABLE_TRAP)) + return ipa_impossible_devirt_target (ie, NULL_TREE); + if (fndecl) + return fndecl; + + /* If not, try to use IPA devirtualization to solve it. */ + ipa_polymorphic_call_context ctx; + if (pmf->outer_type && !pmf->base_via_virtual) + { + ctx.clear_outer_type (pmf->outer_type); + ctx.offset_by (outer_offset * BITS_PER_UNIT); + } + else + { + ctx.clear_outer_type (base); + ctx.offset_by (base_offset * BITS_PER_UNIT); + } + bool final; + vec<cgraph_node *> targets + = possible_polymorphic_call_targets (inner, idx, ctx, &final); + if (final && targets.length () <= 1) + { + if (targets.length () == 1) + return targets[0]->decl; + else + return ipa_impossible_devirt_target (ie, NULL_TREE); + } + return NULL; +} + +/* The same as try_make_edge_direct_simple_call, the call pointer to member + function call version. */ +static struct cgraph_edge * +try_make_edge_direct_ptrmemfunc_call (struct cgraph_edge *ie, + struct ipa_jump_func *jfunc, + tree target_type, + struct cgraph_node *new_root, + class ipa_node_params *new_root_info) +{ + tree pfn = NULL_TREE, delta = NULL_TREE; + cgraph_ptrmemfunc_indirect_info *pmf + = as_a<cgraph_ptrmemfunc_indirect_info *> (ie->indirect_info); + + tree scalar = ipa_value_from_jfunc (new_root_info, jfunc, target_type); + auto recover = [&] (HOST_WIDE_INT offset, tree &val) + { + if (scalar) + val = ipa_find_agg_cst_from_init (scalar, offset, pmf->by_ref); + if (!val) + val = ipa_find_agg_cst_from_jfunc_items (&jfunc->agg, new_root_info, + new_root, offset, pmf->by_ref); + }; + + recover (pmf->pfn_offset, pfn); + if (!pfn) + return NULL; + recover (pmf->delta_offset, delta); + + tree target = ipa_get_ptrmemfunc_direct_target (ie, pfn, delta); + return target ? ipa_make_edge_direct_to_target (ie, target) : NULL; +} + /* Return the target to be used in cases of impossible devirtualization. IE and target (the latter can be NULL) are dumped when dumping is enabled. */ @@ -4577,6 +4747,8 @@ update_indirect_edges_after_inlining (struct cgraph_edge *cs, = dyn_cast <cgraph_polymorphic_indirect_info *> (ie->indirect_info); cgraph_simple_indirect_info *sii = dyn_cast <cgraph_simple_indirect_info *> (ie->indirect_info); + cgraph_ptrmemfunc_indirect_info *pmf + = dyn_cast <cgraph_ptrmemfunc_indirect_info *> (ie->indirect_info); struct ipa_jump_func *jfunc = ipa_get_ith_jump_func (top, param_index); auto_vec<cgraph_node *, 4> spec_targets; @@ -4605,6 +4777,14 @@ update_indirect_edges_after_inlining (struct cgraph_edge *cs, new_root, new_root_info); } + else if (pmf) + { + tree target_type = ipa_get_type (inlined_node_info, param_index); + new_direct_edge + = try_make_edge_direct_ptrmemfunc_call (ie, jfunc, target_type, + new_root, new_root_info); + + } else gcc_unreachable (); @@ -4634,7 +4814,7 @@ update_indirect_edges_after_inlining (struct cgraph_edge *cs, && ipa_get_jf_pass_through_operation (jfunc) == NOP_EXPR) { if (!pii - && sii->agg_contents + && (pmf || sii->agg_contents) && !ipa_get_jf_pass_through_agg_preserved (jfunc)) ie->indirect_info->param_index = -1; else @@ -4655,7 +4835,7 @@ update_indirect_edges_after_inlining (struct cgraph_edge *cs, else if (jfunc->type == IPA_JF_ANCESTOR) { if (!pii - && sii->agg_contents + && (pmf || sii->agg_contents) && !ipa_get_jf_ancestor_agg_preserved (jfunc)) ie->indirect_info->param_index = -1; else @@ -4672,8 +4852,13 @@ update_indirect_edges_after_inlining (struct cgraph_edge *cs, ipa_set_param_used_by_polymorphic_call (new_root_info, param_index, true); } - else + else if (sii) sii->offset += ipa_get_jf_ancestor_offset (jfunc); + else + { + pmf->pfn_offset += ipa_get_jf_ancestor_offset (jfunc); + pmf->delta_offset += ipa_get_jf_ancestor_offset (jfunc); + } } } else @@ -5604,7 +5789,6 @@ ipa_write_indirect_edge_info (struct output_block *ob, { bp = bitpack_create (ob->main_stream); bp_pack_value (&bp, sii->agg_contents, 1); - bp_pack_value (&bp, sii->member_ptr, 1); bp_pack_value (&bp, sii->fnptr_loaded_from_record, 1); bp_pack_value (&bp, sii->by_ref, 1); bp_pack_value (&bp, sii->guaranteed_unmodified, 1); @@ -5621,6 +5805,21 @@ ipa_write_indirect_edge_info (struct output_block *ob, streamer_write_uhwi (ob, sii->fld_offset); } } + else if (cgraph_ptrmemfunc_indirect_info *pmf + = dyn_cast <cgraph_ptrmemfunc_indirect_info *> (cs->indirect_info)) + { + bp = bitpack_create (ob->main_stream); + bp_pack_value (&bp, pmf->by_ref, 1); + bp_pack_value (&bp, pmf->base_via_virtual, 1); + streamer_write_bitpack (&bp); + + streamer_write_hwi (ob, pmf->param_index); + streamer_write_hwi (ob, pmf->pfn_offset); + streamer_write_hwi (ob, pmf->delta_offset); + stream_write_tree (ob, pmf->method_type, true); + stream_write_tree (ob, pmf->outer_type, true); + streamer_write_hwi (ob, pmf->base_offset); + } else gcc_assert (cs->indirect_info->param_index == -1); } @@ -5664,7 +5863,6 @@ ipa_read_indirect_edge_info (class lto_input_block *ib, { bp = streamer_read_bitpack (ib); sii->agg_contents = bp_unpack_value (&bp, 1); - sii->member_ptr = bp_unpack_value (&bp, 1); sii->fnptr_loaded_from_record = bp_unpack_value (&bp, 1); sii->by_ref = bp_unpack_value (&bp, 1); sii->guaranteed_unmodified = bp_unpack_value (&bp, 1); @@ -5682,6 +5880,23 @@ ipa_read_indirect_edge_info (class lto_input_block *ib, if (info && sii->param_index >= 0) ipa_set_param_used_by_indirect_call (info, sii->param_index, true); } + else if (cgraph_ptrmemfunc_indirect_info *pmf + = dyn_cast <cgraph_ptrmemfunc_indirect_info *> (cs->indirect_info)) + { + bp = streamer_read_bitpack (ib); + pmf->by_ref = bp_unpack_value (&bp, 1); + pmf->base_via_virtual = bp_unpack_value (&bp, 1); + + pmf->param_index = (int) streamer_read_hwi (ib); + pmf->pfn_offset = (HOST_WIDE_INT) streamer_read_hwi (ib); + pmf->delta_offset = (HOST_WIDE_INT) streamer_read_hwi (ib); + pmf->method_type = stream_read_tree (ib, data_in); + pmf->outer_type = stream_read_tree (ib, data_in); + pmf->base_offset = (HOST_WIDE_INT) streamer_read_hwi (ib); + + if (info && pmf->param_index >= 0) + ipa_set_param_used_by_indirect_call (info, pmf->param_index, true); + } else cs->indirect_info->param_index = -1; } diff --git a/gcc/ipa-prop.h b/gcc/ipa-prop.h index 9099afaea30..04365bb8f20 100644 --- a/gcc/ipa-prop.h +++ b/gcc/ipa-prop.h @@ -1202,7 +1202,8 @@ tree ipa_get_indirect_edge_target (struct cgraph_edge *ie, struct cgraph_edge *ipa_make_edge_direct_to_target (struct cgraph_edge *, tree, bool speculative = false); tree ipa_impossible_devirt_target (struct cgraph_edge *, tree); - +tree ipa_get_ptrmemfunc_direct_target (struct cgraph_edge *ie, tree pfn_tree, + tree delta_tree); /* Functions related to both. */ void ipa_analyze_node (struct cgraph_node *); diff --git a/gcc/lto-cgraph.cc b/gcc/lto-cgraph.cc index 9a71110d5c2..dd902c947be 100644 --- a/gcc/lto-cgraph.cc +++ b/gcc/lto-cgraph.cc @@ -1618,6 +1618,10 @@ input_edge (class lto_input_block *ib, vec<symtab_node *> nodes, edge->indirect_info = (new (ggc_alloc<cgraph_simple_indirect_info> ()) cgraph_simple_indirect_info (ecf_flags)); + else if (ii_kind == CIIK_PTRMEMFUNC) + edge->indirect_info + = (new (ggc_alloc<cgraph_ptrmemfunc_indirect_info> ()) + cgraph_ptrmemfunc_indirect_info (ecf_flags)); else edge->indirect_info = (new (ggc_alloc<cgraph_indirect_call_info> ()) diff --git a/gcc/testsuite/g++.dg/opt/devirtpmf6.C b/gcc/testsuite/g++.dg/opt/devirtpmf6.C new file mode 100644 index 00000000000..b7face26577 --- /dev/null +++ b/gcc/testsuite/g++.dg/opt/devirtpmf6.C @@ -0,0 +1,62 @@ +// IPA devirtualizable tests +// { dg-do compile { target c++11 } } +// { dg-options "-O2" } + +struct A { virtual void abcabc () = 0; }; +struct B : A { void abcabc () final; }; + +// { dg-final { scan-assembler "_ZN1B6abcabcEv" } } +void foo1 (B *obj, void (B::* mem) ()) { (obj->*mem) (); } +void bar1 (B *obj) { foo1 (obj, &B::abcabc); } + + +struct F { virtual void abcabc () final; }; +struct G { virtual void defdef () final; }; +struct H : F, G {}; + +// { dg-final { scan-assembler "_ZN1G6defdefEv" } } +void foo2 (H *obj, void (H::* mem) ()) { (obj->*mem) (); } +void bar2 (H *obj) { foo2 (obj, &G::defdef); } + + +struct S +{ + int x; + virtual void f (int); +}; +struct T +{ + int y; + virtual void g (int); + virtual void h (int); +}; +struct U final : S, T {}; + +// { dg-final { scan-assembler "_ZN1T1hEi" } } +void foo3 (U *obj, void (U::* mem) (int), int bar) { return (obj->*mem) (bar); } +void bar3 (U *obj, int val) { return foo3 (obj, &U::h, val); } + + +struct W +{ + virtual void x () = 0; + virtual void y () = 0; +}; +struct X : virtual W { void x () final; }; +struct Y : virtual W { void y () final; }; +struct Z final : X, Y {}; +struct PMF +{ + int x; + void (Z::*mem) (); +}; + +// { dg-final { scan-assembler-times "_ZN1Y1yEv" 2 } } +void +foo4 (Z *obj[2], void (Z::* mem) ()) +{ + (obj[0]->*mem) (); + (obj[1]->*mem) (); +} +void bar4 (Z *obj[2], PMF pmf) { foo4 (obj, pmf.mem); } +void baz4 (Z *obj[2]) { bar4 (obj, PMF{10, &Z::y}); } diff --git a/gcc/testsuite/g++.dg/opt/devirtpmf7.C b/gcc/testsuite/g++.dg/opt/devirtpmf7.C new file mode 100644 index 00000000000..699804d23f1 --- /dev/null +++ b/gcc/testsuite/g++.dg/opt/devirtpmf7.C @@ -0,0 +1,23 @@ +// IPA inline impossible devirtualize tests +// { dg-do compile { target c++11 } } +// { dg-options "-O3 -fdump-ipa-inline-details" } +// { dg-final { scan-ipa-dump "__builtin_unreachable" "inline" } } + +struct A { virtual void foo (int); }; +struct B { virtual void bar (int); }; +struct C : A, B +{ + virtual void foo (int) final; + virtual void bar (int) final; +}; +struct D : B { virtual void bar2 (int); }; + +void foo (C *obj, void (C::* mem) (int)) { (obj->*mem) (1); } +void +bar1 (C *obj) +{ + void (C::*mem) (int) = static_cast<void (C::*) (int)> + (static_cast<void (B::*) (int)> (&D::bar2)); + foo (obj, mem); +} +void bar2 (C *obj) { foo (obj, nullptr); } diff --git a/gcc/testsuite/g++.dg/opt/devirtpmf8.C b/gcc/testsuite/g++.dg/opt/devirtpmf8.C new file mode 100644 index 00000000000..f01fb68aa8d --- /dev/null +++ b/gcc/testsuite/g++.dg/opt/devirtpmf8.C @@ -0,0 +1,28 @@ +// IPA-CP tests +// { dg-do compile { target c++11 } } +// { dg-options "-O3 -fdump-ipa-cp --param ipa-cp-eval-threshold=1" } +// { dg-final { scan-ipa-dump "Discovered an indirect call to a known target" "cp" } } +// { dg-final { scan-assembler-times "_ZN1B6abcabcEv" 2 } } + +struct A +{ + virtual void abcabc () = 0; +}; + +struct B : A +{ + void abcabc () final; +}; + +void +foo (B *b, void (B::*mem) ()) { (b->*mem) (); } +[[gnu::noinline]] void +foo_no_inline (B *b, void (B::*mem) ()) { (b->*mem) (); } + +void +bar (B *b) +{ + foo (b, &B::abcabc); + foo_no_inline (b, &B::abcabc); +} + -- 2.43.0