[PATCH] [AutoFDO][IPA-CP] Handle zero-caller locals after AutoFDO early inlining

AutoFDO early inlining can inline all direct calls to a local
function while speculative call promotion keeps the function
address-taken.  The function consequently remains in the call graph
with no direct callers, triggering the -ftoplevel-reorder checking
assertion in IPA-CP.

Ordinary IPA devirtualization does not normally produce this state at
IPA-CP: it creates speculative edges before IPA-CP, but those edges are
inlined only by the later IPA inliner.  IPA-CP therefore still sees a
direct caller.  AutoFDO instead performs speculative inlining early, so
only its IPA_REF_ADDR may remain by the time IPA-CP runs.

When AutoFDO inlining is enabled, recognize zero-caller functions that
cannot be removed after their direct calls disappear and initialize
their lattices as variable.  Preserve the assertion for removable
locals and non-AutoFDO compilation.

gcc/ChangeLog:

	* ipa-cp.cc (initialize_node_lattices): Handle zero-caller local
	functions left after AutoFDO early inlining.

Regression tested on aarch64-linux-gnu with no new regressions

Signed-off-by: Kugan Vivekanandarajah <[email protected]>
---
 gcc/ipa-cp.cc | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/gcc/ipa-cp.cc b/gcc/ipa-cp.cc
index c1b1f6201fd3..d68952ec8711 100644
--- a/gcc/ipa-cp.cc
+++ b/gcc/ipa-cp.cc
@@ -1443,7 +1443,18 @@ initialize_node_lattices (struct cgraph_node *node)
 						  NULL, true);
       else if (caller_count == 0)
 	{
-	  gcc_checking_assert (!opt_for_fn (node->decl, flag_toplevel_reorder));
+	  /* With -ftoplevel-reorder, a local body with no direct callers
+	     should already have been removed.  AutoFDO early-inline of a
+	     speculative call expands via gimple_ic (&target), so after all
+	     direct call edges are inlined the local can remain address-taken
+	     with unknown indirect uses; treat that as VARIABLE.  Ordinary
+	     IPA devirtualization does not need this exception: it creates
+	     speculative edges before IPA-CP, but inlines them only later, so
+	     IPA-CP still sees a direct caller.  */
+	  if (!(flag_auto_profile && flag_auto_profile_inlining
+		&& !node->can_remove_if_no_direct_calls_p (false)))
+	    gcc_checking_assert
+	      (!opt_for_fn (node->decl, flag_toplevel_reorder));
 	  variable = true;
 	}
     }
-- 
2.34.1