Re: [PATCH] c++, v2: Reject UB in static_cast downcasts during constant evaluation

On Wed, Aug 05, 2026 at 03:49:18PM -0400, Jason Merrill wrote:
> > --- gcc/fold-const.cc.jj	2026-08-04 17:00:02.424807528 +0200
> > +++ gcc/fold-const.cc	2026-08-05 14:53:17.951727844 +0200
> > @@ -9429,6 +9429,14 @@ fold_unary_loc (location_t loc, enum tre
> >   	      && TREE_CODE (TREE_TYPE (arg00)) != REFERENCE_TYPE)
> >   	    return NULL_TREE;
> > +	  if (RECORD_OR_UNION_TYPE_P (TREE_TYPE (type))
> > +	      && RECORD_OR_UNION_TYPE_P (TREE_TYPE (TREE_TYPE (arg0)))
> > +	      && (TYPE_MAIN_VARIANT (TREE_TYPE (type))
> > +		  != TYPE_MAIN_VARIANT (TREE_TYPE (TREE_TYPE (arg0))))
> > +	      && !in_gimple_form
> > +	      && lang_GNU_CXX ())
> > +	    return NULL_TREE;
> > +
> >   	  arg00 = fold_convert_loc (loc, type, arg00);
> >   	  return fold_build_pointer_plus_loc (loc, arg00, arg01);
> >   	}
> > with a comment, then the disadvantage is that it will not optimize even when
> > doing cp_fold* or during cp_gimplify_expr.
> 
> Yes, that's what I was thinking about.
> 
> > That said, I've been wondering if such change would prevent optimization of
> > say:
> > struct A { int a; };
> > #ifdef __cplusplus
> > struct B : A { int b; };
> > struct C : B { int c; };
> > #else
> > struct C { int c; };
> > #endif
> > 
> > A *
> > foo (A *p)
> > {
> >    return ((A *) (((C *) ((A *) (((C *) ((A *) (((C *) ((A *) (((C *) p) + 1))) + 1))) + 1))) + 1));
> > }
> > Turns out it doesn't prevent that, because match.pd has also
> > /* Associate (p +p off1) +p off2 as (p +p (off1 + off2)).  */
> > (simplify
> >    (pointer_plus (pointer_plus:s @0 @1) @3)
> >    (pointer_plus @0 (plus @1 @3)))
> > #if GENERIC
> > (simplify
> >    (pointer_plus (convert:s (pointer_plus:s @0 @1)) @3)
> >    (pointer_plus (convert:type @0) (plus @1 @3)))
> > #endif
> > optimization where the GENERIC only simplification clearly
> > looks also inapproprite for C++ constant evaluation.  But surprisingly
> > commenting that out doesn't help, the first one triggers there too,
> > so wonder what else gets rid of the casts in there.
> Another possible approach to this issue would be for
> cxx_fold_pointer_plus_expression to handle the case of a NOP_EXPR on the lhs
> specially so it isn't exposed to the usual NOP_EXPR handling?

I'd be afraid that any extra folding before cp_fold can turn into
accepts-invalid if the casts are completely optimized away.
Though, so far I've been unable to construct a testcase that would
trigger it.  E.g. on the following in the first case the thing is
that the casts are added by build_static_cast_1 -> build_base_path
which does e.g.
expr = build1 (NOP_EXPR, ptr_target_type, expr);
rathern than cp_fold_convert, and then the cp_fold_convert does nothing.
And in the bar case below the POINTER_PLUS_EXPR in there is conditional
on the pointer not being null and also isn't being optimized out.

Guess I'll try the cxx_fold_pointer_plus_expression change you're proposing.

struct A {};
struct B : A {};
struct C { int c; };
struct D { int d; };
struct E : C, D { int e; }; 

constexpr bool
foo (bool x)
{
  struct A a = {};
  A *b = &a;
  A *c = nullptr;
  if (x)
    {
      c = static_cast <A *> (static_cast <B *> (static_cast <A *> (static_cast <B *> (b))));
    }
  return true;
}

constexpr bool a = foo (true);

constexpr bool
bar (bool x)
{
  struct D a = { 42 };
  D *b = &a;
  D *c = nullptr;
  if (x)
    {
      c = static_cast <D *> (static_cast <E *> (static_cast <D *> (static_cast <E *> (b))));
    }
  return true;
}

constexpr bool b = bar (true);


	Jakub