[RFC 1/3] gdb: limited support for frame cache flushing while building stack

This commit adds a mechanism to GDB to allow for a limit number of
frame cache flushes while GDB is building the inferior's stack.

Normally GDB takes care to ensure that the frame cache is never
flushed while building the inferior's stack, i.e. while creating new
frames.  If we did allow flushing the frame cache at will during frame
creation then we would (potentially) never finish building the stack,
as each frame cache flush requires that we restart building the stack
from the start.

The problem we have, is that as we add more Python API hooks into GDB,
especially around stack unwinding, the more chance we have that a user
action is going to trigger a frame cache flush.  We have already seen
at least one bug in this area PR gdb/32120 where an apparently
innocent RemoteTargetConnection.send_packet call can trigger a frame
cache flush.

In the next commit I am adding support for deferred debug information
downloading via debuginfod.  With that commit we will initially
download the .gdb_index for a debug info file, and only download the
full debug info file later when GDB realises that it needs it.

The problem that this cases is that GDB might realise that it needs a
debug info file while it is building the stack.  For example if frame
5 is in a shared library for which we are using deferred debug info
downloading, then GDB will now download the required debug info file
until it tries to build frame 5.  Loading the deferred debug info file
will trigger the new_objfile observer, which will then load any
associated Python scripts.  These Python scripts could trigger a frame
cache flush, e.g. by installing a custom stack unwinder.

The change presented here is designed specifically to handle the
deferred debug info download case.  In that case we only expect to see
a frame cache flush the first time we attempt to create a frame; the
debug info is downloaded, the support scripts are loaded, and the
frame cache is flushed.  If we try to create the frame again, then the
debug information is already downloaded and added to GDB so we no
longer expect to see a second debug info download (for the same
frame), and so there should be no new objfile event, and no possible
frame cache flush.

This is achieved by adding a mechanism to defer the frame cache
flushing, scoped_defer_reinit_frame_cache.  While a
scoped_defer_reinit_frame_cache is active on the stack, any frame
cache flushes are postponed, this means that the frame_info objects
will not be invalidated, and frame creation can continue as if the
flush had not happened.  This does mean that the generated frame could
be invalid though.  When the scoped_defer_reinit_frame_cache goes out
of scope then any pending frame cache flushes are performed,
invalidating any (potentially) invalid frames that have been created.

In addition to the above a new helper function,
with_protected_frame_cache has been added.  This helper wraps around
some other function call, performing the wrapped function call with a
scoped_defer_reinit_frame_cache in place.  After calling the wrapped
function, if there is no frame cache flush pending then we are done.

If there is a pending frame cache flush then the wrapped function is
called a second time.  Again, after this call, if there are no pending
frame cache flushes, then we are done.  But, if after the second call
there is still a pending flush, then an error is thrown.  This
indicates the case where (apparently) the frame cache is being flushed
every time GDB tries to build a frame.  We could imagine situations
where exactly 3 attempts is needed to reach a stable situation.  Or 4
attempts.  Or 5.  Or any number we choose.  The problem is we cannot
just spin forever hoping that GDB eventually breaks out of the loop.
At some point we need to just give up.  I believe debuginfod offers a
compelling case for making 2 attempts, so that's what this patch
proposes.

In order to support the new with_protected_frame_cache template
function there's a new type trait which can be used to check that
every argument in an argument pack is const.  This is needed because,
if the function with_protected_frame_cache is wrapping can update its
arguments, then calling the wrapped function twice might not produce
the expected results.
---
 gdb/frame.c                                   | 124 +++++++++++++++++-
 .../gdb.python/py-frame-cache-flushing-lib1.c |  26 ++++
 .../gdb.python/py-frame-cache-flushing-lib2.c |  26 ++++
 .../gdb.python/py-frame-cache-flushing.c      |  55 ++++++++
 .../gdb.python/py-frame-cache-flushing.exp    | 115 ++++++++++++++++
 .../gdb.python/py-frame-cache-flushing.py     |  52 ++++++++
 gdbsupport/traits.h                           |   7 +
 7 files changed, 402 insertions(+), 3 deletions(-)
 create mode 100644 gdb/testsuite/gdb.python/py-frame-cache-flushing-lib1.c
 create mode 100644 gdb/testsuite/gdb.python/py-frame-cache-flushing-lib2.c
 create mode 100644 gdb/testsuite/gdb.python/py-frame-cache-flushing.c
 create mode 100644 gdb/testsuite/gdb.python/py-frame-cache-flushing.exp
 create mode 100644 gdb/testsuite/gdb.python/py-frame-cache-flushing.py

diff --git a/gdb/frame.c b/gdb/frame.c
index b91e18fad99..912404cd26a 100644
--- a/gdb/frame.c
+++ b/gdb/frame.c
@@ -97,6 +97,14 @@ static frame_info_ptr selected_frame;
 
 static frame_info_ptr sentinel_frame;
 
+/* When nonzero, reinit_frame_cache is deferred: frame_info objects
+   are kept alive so that mid-operation code (e.g. the DWARF unwinder
+   computing a frame ID) does not encounter freed memory.  */
+static unsigned int defer_reinit_frame_cache_depth = 0;
+
+/* Set when reinit_frame_cache was requested while deferred.  */
+static bool defer_reinit_frame_cache_pending = false;
+
 /* See frame.h.  */
 
 unsigned int
@@ -424,6 +432,100 @@ scoped_restore_selected_frame::~scoped_restore_selected_frame ()
   set_language (m_lang);
 }
 
+/* RAII class to defer reinit_frame_cache calls.  While an instance of
+   this class is alive, calls to reinit_frame_cache are deferred: the
+   frame cache is not cleared and frame_info objects remain valid.
+   When the last instance goes out of scope, a single
+   reinit_frame_cache call is made if any were deferred.
+
+   This is used during deferred debuginfo downloads to prevent the
+   frame cache from being destroyed mid-operation (e.g. while the
+   DWARF unwinder is computing a frame ID).  */
+
+class scoped_defer_reinit_frame_cache
+{
+public:
+  scoped_defer_reinit_frame_cache ()
+  {
+    defer_reinit_frame_cache_depth++;
+  }
+
+  ~scoped_defer_reinit_frame_cache ()
+  {
+    gdb_assert (defer_reinit_frame_cache_depth > 0);
+    defer_reinit_frame_cache_depth--;
+
+    if (defer_reinit_frame_cache_depth == 0
+	&& defer_reinit_frame_cache_pending)
+    {
+      frame_debug_printf ("performing deferred frame cache reinit");
+      defer_reinit_frame_cache_pending = false;
+      reinit_frame_cache ();
+    }
+  }
+
+  bool will_trigger_reinit () const
+  {
+    gdb_assert (defer_reinit_frame_cache_depth > 0);
+    return this->has_pending_reinit () && defer_reinit_frame_cache_depth == 1;
+  }
+
+  bool has_pending_reinit () const
+  {
+    gdb_assert (defer_reinit_frame_cache_depth > 0);
+    return defer_reinit_frame_cache_pending;
+  }
+
+  DISABLE_COPY_AND_ASSIGN (scoped_defer_reinit_frame_cache);
+};
+
+/* Call Func passing in Args while a scoped_defer_reinit_frame_cache is in
+   effect.  Once Func completes, if the frame cache has been reinitialised
+   then try calling Func again.  If after the second call the frame cache
+   has again been initialised then raise an error.  Because Func can be
+   called multiple times, it is required that every argument in ARGS be
+   'const'.
+
+   This can be used to wrap frame unwinding related calls where an
+   extension language hook might trigger a frame cache flush.  The hope is
+   that whatever action triggers the flush will only happen the first
+   time, and that the second time through will not result in a frame cache
+   flush.  */
+
+template <typename Func, typename... Args,
+	  typename = gdb::Requires<gdb::all_args_are_const<Args...>>>
+static decltype(auto)
+with_protected_frame_cache (Func&& func, Args&&... args)
+{
+  FRAME_SCOPED_DEBUG_ENTER_EXIT;
+
+  using ReturnType = std::invoke_result_t<Func, Args...>;
+
+  for (int i = 0; i < 2; ++i)
+    {
+      scoped_defer_reinit_frame_cache defer_reinit_frame_cache;
+
+      if constexpr (std::is_void_v<ReturnType>)
+	{
+	  std::invoke (std::forward<Func> (func),
+		       std::forward<Args> (args)...);
+
+	  if (!defer_reinit_frame_cache.will_trigger_reinit ())
+	    return;
+	}
+      else
+	{
+	  decltype(auto) result = std::invoke (std::forward<Func> (func),
+					       std::forward<Args> (args)...);
+
+	  if (!defer_reinit_frame_cache.will_trigger_reinit ())
+	    return result;
+	}
+    }
+
+  error ("frame cache repeatedly reinitialized");
+}
+
 /* Flag to control debugging.  */
 
 bool frame_debug;
@@ -1985,8 +2087,8 @@ invalidate_selected_frame ()
 
 /* See frame.h.  */
 
-void
-select_frame (const frame_info_ptr &fi)
+static void
+select_frame_1 (const frame_info_ptr &fi)
 {
   gdb_assert (fi != nullptr);
 
@@ -2062,6 +2164,12 @@ select_frame (const frame_info_ptr &fi)
     }
 }
 
+void
+select_frame (const frame_info_ptr &fi)
+{
+  with_protected_frame_cache (select_frame_1, fi);
+}
+
 /* Create an arbitrary (i.e. address specified by user) or innermost frame.
    Always returns a non-NULL value.  */
 
@@ -2169,6 +2277,15 @@ frame_observer_target_changed (struct target_ops *target)
 void
 reinit_frame_cache (void)
 {
+  if (defer_reinit_frame_cache_depth > 0)
+    {
+      frame_debug_printf ("mark frame cache flush as pending");
+      defer_reinit_frame_cache_pending = true;
+      return;
+    }
+
+  frame_debug_printf ("flushing the frame cache");
+
   ++frame_cache_generation;
 
   if (!frame_stash.empty ())
@@ -2551,7 +2668,8 @@ get_prev_frame_always (const frame_info_ptr &this_frame)
 
   try
     {
-      prev_frame = get_prev_frame_always_1 (this_frame);
+      prev_frame = with_protected_frame_cache (get_prev_frame_always_1,
+					       this_frame);
     }
   catch (const gdb_exception_error &ex)
     {
diff --git a/gdb/testsuite/gdb.python/py-frame-cache-flushing-lib1.c b/gdb/testsuite/gdb.python/py-frame-cache-flushing-lib1.c
new file mode 100644
index 00000000000..b954375fe50
--- /dev/null
+++ b/gdb/testsuite/gdb.python/py-frame-cache-flushing-lib1.c
@@ -0,0 +1,26 @@
+/* This testcase is part of GDB, the GNU debugger.
+
+   Copyright 2026 Free Software Foundation, Inc.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+/* ... */
+
+typedef void (*callback_t) (void);
+
+void
+library1_function (callback_t cb)
+{
+  cb ();
+}
diff --git a/gdb/testsuite/gdb.python/py-frame-cache-flushing-lib2.c b/gdb/testsuite/gdb.python/py-frame-cache-flushing-lib2.c
new file mode 100644
index 00000000000..b742f2c4347
--- /dev/null
+++ b/gdb/testsuite/gdb.python/py-frame-cache-flushing-lib2.c
@@ -0,0 +1,26 @@
+/* This testcase is part of GDB, the GNU debugger.
+
+   Copyright 2026 Free Software Foundation, Inc.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+/* ... */
+
+typedef void (*callback_t) (void);
+
+void
+library2_function (callback_t cb)
+{
+  cb ();
+}
diff --git a/gdb/testsuite/gdb.python/py-frame-cache-flushing.c b/gdb/testsuite/gdb.python/py-frame-cache-flushing.c
new file mode 100644
index 00000000000..5f169d64751
--- /dev/null
+++ b/gdb/testsuite/gdb.python/py-frame-cache-flushing.c
@@ -0,0 +1,55 @@
+/* This testcase is part of GDB, the GNU debugger.
+
+   Copyright 2026 Free Software Foundation, Inc.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
+
+/* ... */
+
+typedef void (*callback_t) (void);
+extern void library1_function (callback_t cb);
+extern void library2_function (callback_t cb);
+
+volatile int global_var;
+
+void
+f0 (void)
+{
+  global_var = 42;	/* Break here.  */
+}
+
+void
+f1 (void)
+{
+  f0 ();
+}
+
+void
+f3 (void)
+{
+  library2_function (f1);
+}
+
+void
+f5 (void)
+{
+  library1_function (f3);
+}
+
+int
+main (void)
+{
+  f5 ();
+  return global_var - 42;
+}
diff --git a/gdb/testsuite/gdb.python/py-frame-cache-flushing.exp b/gdb/testsuite/gdb.python/py-frame-cache-flushing.exp
new file mode 100644
index 00000000000..988a34e66e4
--- /dev/null
+++ b/gdb/testsuite/gdb.python/py-frame-cache-flushing.exp
@@ -0,0 +1,115 @@
+# Copyright 2026 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# ....
+
+load_lib gdb-python.exp
+
+standard_testfile .c -lib1.c -lib2.c
+
+# Build the first shared library.
+set lib1_testfile "lib1-${testfile}.so"
+set lib1_srcfile $srcfile2
+set lib1_binfile [standard_output_file $lib1_testfile]
+if { [build_executable "build $lib1_testfile" $lib1_testfile $lib1_srcfile \
+	  {debug build-id shlib}] } {
+    return
+}
+
+# Build the second shared library.
+set lib2_testfile "lib2-${testfile}.so"
+set lib2_srcfile $srcfile3
+set lib2_binfile [standard_output_file $lib2_testfile]
+if { [build_executable "build $lib2_testfile" $lib2_testfile $lib2_srcfile \
+	  {debug build-id shlib}] } {
+    return
+}
+
+if { [build_executable "build executable" $testfile $srcfile \
+	  [list debug build-id shlib=$lib1_binfile \
+	       shlib=$lib2_binfile]] != 0 } {
+    return
+}
+
+set remote_python_file [gdb_remote_download host ${srcdir}/${subdir}/${testfile}.py]
+
+clean_restart
+
+proc test { flush_levels } {
+    clean_restart $::testfile
+
+    if {![runto_main]} {
+	return
+    }
+
+    gdb_test_no_output "source $::remote_python_file" "load python file"
+
+    set expected_failure_level -1
+    set counter 0
+    foreach lvl $flush_levels {
+	incr counter
+	with_test_prefix "setup step $counter" {
+	    gdb_test_no_output "python flush_cache_at_levels\[$lvl\] \
+				  = flush_cache_at_levels.get($lvl, 0) + 1" \
+		"setup step $counter, prepare for cache flush at level $lvl"
+	    set times [get_python_valueof "flush_cache_at_levels\[$lvl\]" \
+			   UNKNOWN "get flush count for level $lvl"]
+	    if { $times > 1 && ($lvl < $expected_failure_level \
+				    || $expected_failure_level == -1) } {
+		set expected_failure_level $lvl
+	    }
+	}
+    }
+
+    gdb_breakpoint [gdb_get_line_number "Break here." $::srcfile]
+
+    gdb_continue_to_breakpoint "breakpoint in f0"
+
+    set full_bt_regexp \
+	[list "#0  f0 \\(\\) at \[^\r\n\]+" \
+	      "#1  $::hex in f1 \\(\\) at \[^\r\n\]+" \
+	      "#2  $::hex in library2_function \\(cb=$::hex <f1>\\) at \[^\r\n\]+" \
+	      "#3  $::hex in f3 \\(\\) at \[^\r\n\]+" \
+	      "#4  $::hex in library1_function \\(cb=$::hex <f3>\\) at \[^\r\n\]+" \
+	      "#5  $::hex in f5 \\(\\) at \[^\r\n\]+" \
+	      "#6  $::hex in main \\(\\) at \[^\r\n\]+"]
+
+    set lvl 0
+    set re {}
+    foreach line $full_bt_regexp {
+	if { $lvl == $expected_failure_level } {
+	    lappend re "frame cache repeatedly reinitialized"
+	    break
+	}
+
+	lappend re $line
+	incr lvl
+    }
+
+    if { [llength $re] > 1 } {
+	set re [multi_line {*}$re]
+    } else {
+	set re [lindex $re 0]
+    }
+
+    gdb_test "bt" $re "check backtrace"
+}
+
+# Each LEVEL_LIST is a list of the frame levels at which the frame
+# cache should be flushed.  The frame cache will be flush once for
+# each occurance of a frame level within a list.
+foreach_with_prefix level_list { {} {0} {1} {2} {3} {4} {5} {6} {2 4} {0 1 2 3 4 5 6} {4 4} } {
+    test $level_list
+}
diff --git a/gdb/testsuite/gdb.python/py-frame-cache-flushing.py b/gdb/testsuite/gdb.python/py-frame-cache-flushing.py
new file mode 100644
index 00000000000..73644d4110b
--- /dev/null
+++ b/gdb/testsuite/gdb.python/py-frame-cache-flushing.py
@@ -0,0 +1,52 @@
+# Copyright (C) 2026 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import gdb
+from gdb.unwinder import Unwinder
+
+test_debugging = False
+
+
+def debug_log(msg):
+    global test_debugging
+    if not test_debugging:
+        return
+    print(msg)
+
+
+flush_cache_at_levels = {}
+
+
+class cache_flush_unwinder(Unwinder):
+    def __init__(self):
+        super().__init__("cache_flush_unwinder")
+
+    def __call__(self, pending_frame):
+        global flush_cache_at_levels
+
+        level = pending_frame.level()
+        debug_log("Cache flushing unwinder at level %d" % (level))
+
+        if level in flush_cache_at_levels:
+            if flush_cache_at_levels[level] > 0:
+                debug_log(" '-> Flushing the frame cache")
+                gdb.invalidate_cached_frames()
+                flush_cache_at_levels[level] -= 1
+
+        # This unwinder never claims any frames.
+        return None
+
+
+gdb.unwinder.register_unwinder(None, cache_flush_unwinder(), True)
diff --git a/gdbsupport/traits.h b/gdbsupport/traits.h
index 4fe05ee0e0e..be738186da2 100644
--- a/gdbsupport/traits.h
+++ b/gdbsupport/traits.h
@@ -104,6 +104,13 @@ using And = std::conjunction<T...>;
 /* Concepts-light-like helper to make SFINAE logic easier to read.  */
 template<typename Condition>
 using Requires = typename std::enable_if<Condition::value, void>::type;
+
+/* Type trait that can be used to ensure an argument pack contains only
+   constant arguments.  */
+template <typename... Args>
+struct all_args_are_const
+  : std::bool_constant<(std::is_const_v<std::remove_reference_t<Args>> && ...)>
+{ /* Nothing.  */ };
 }
 
 template<typename T>
-- 
2.25.4