Re: [PATCH] c++, v2: Reject UB in static_cast downcasts during constant evaluation

On Thu, Jul 30, 2026 at 09:04:59PM -0400, Jason Merrill wrote:
> > Here is an updated patch, which
> > 1) has slightly different wording mostly as you suggested,
> >     I have just used the pointed or referenced type rather than a pointer
> >     (what would be printed in the reference case anyway?)
> > 2) in cp_fold_convert I've stopped using fold_convert for the
> >     INDIRECT_TYPE_P to INDIRECT_TYPE_P casts so that we avoid the
> >     Convert (T1)(X p+ Y) into ((T1)X p+ Y), for pointer type, when ...
> >     problematic fold-const.cc optimization, instead it optimizes just
> >     some simple cases
> > 3) had to move -fsanitize=null checking for references on nullptr in order
> >     not to regress one ubsan testcase
> > 
> > Bootstrapped successfull on both x86_64-linux and i686-linux, but has one
> > regression on both:
> > FAIL: std/ranges/adaptors/slide/1.cc  -std=gnu++23 (test for excess errors)
> > Excess errors:
> > /home/jakub/src/gcc/libstdc++-v3/testsuite/std/ranges/adaptors/slide/1.cc:110: error: non-constant condition for static assertion
> > /home/jakub/src/gcc/obj74/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/ranges_base.h:1016: error: '(((((int*)(& x)) + 12) - (((int*)(& x)) + 4)) / 4)' is not a constant expression
> > and ditto with -std=gnu++29.  I'm afraid something relies on
> > some of the fold_convert optimizations, dunno if we should try harder
> > during POINTER_DIFF_EXPR folding or what.
> What if we disable the fold_unary_loc transformation if the pointers are to
> different RECORD_TYPEs?  There are already a couple other exceptions.

Note, with the r17-2979 commit in, the patch as posted doesn't cause
regressions.  It can be also narrowed, so in cvt.cc do fold_convert
also if not
+	  && CLASS_TYPE_P (TREE_TYPE (type))
+	  && CLASS_TYPE_P (TREE_TYPE (TREE_TYPE (expr)))
(but TYPE_MAIN_VARIANT check in that case doesn't really work,
we have there cases we don't want to optimize where
TREE_TYPE (type) and TREE_TYPE (TREE_TYPE (expr)) is the same type,
but after stripping nops from expr we end up with some problematic type.

Regarding the suggested fold-const.cc change, I thought
"Convert (T1)(X p+ Y) into ((T1)X p+ Y), for pointer type, when the new"
is a pretty important optimization that allows moving around
POINTER_PLUS_EXPRs so that they can be optimized together.
If you mean
--- gcc/fold-const.cc.jj	2026-08-04 17:00:02.424807528 +0200
+++ gcc/fold-const.cc	2026-08-05 14:53:17.951727844 +0200
@@ -9429,6 +9429,14 @@ fold_unary_loc (location_t loc, enum tre
 	      && TREE_CODE (TREE_TYPE (arg00)) != REFERENCE_TYPE)
 	    return NULL_TREE;
 
+	  if (RECORD_OR_UNION_TYPE_P (TREE_TYPE (type))
+	      && RECORD_OR_UNION_TYPE_P (TREE_TYPE (TREE_TYPE (arg0)))
+	      && (TYPE_MAIN_VARIANT (TREE_TYPE (type))
+		  != TYPE_MAIN_VARIANT (TREE_TYPE (TREE_TYPE (arg0))))
+	      && !in_gimple_form
+	      && lang_GNU_CXX ())
+	    return NULL_TREE;
+
 	  arg00 = fold_convert_loc (loc, type, arg00);
 	  return fold_build_pointer_plus_loc (loc, arg00, arg01);
 	}
with a comment, then the disadvantage is that it will not optimize even when
doing cp_fold* or during cp_gimplify_expr.
That said, I've been wondering if such change would prevent optimization of
say:
struct A { int a; };
#ifdef __cplusplus
struct B : A { int b; };
struct C : B { int c; };
#else
struct C { int c; };
#endif

A *
foo (A *p)
{
  return ((A *) (((C *) ((A *) (((C *) ((A *) (((C *) ((A *) (((C *) p) + 1))) + 1))) + 1))) + 1));
}
Turns out it doesn't prevent that, because match.pd has also
/* Associate (p +p off1) +p off2 as (p +p (off1 + off2)).  */
(simplify
  (pointer_plus (pointer_plus:s @0 @1) @3)
  (pointer_plus @0 (plus @1 @3)))
#if GENERIC
(simplify
  (pointer_plus (convert:s (pointer_plus:s @0 @1)) @3)
  (pointer_plus (convert:type @0) (plus @1 @3)))
#endif
optimization where the GENERIC only simplification clearly
looks also inapproprite for C++ constant evaluation.  But surprisingly
commenting that out doesn't help, the first one triggers there too,
so wonder what else gets rid of the casts in there.

	Jakub