Re: [PATCH] c++, v2: Reject UB in static_cast downcasts during constant evaluation

On 8/5/26 9:58 AM, Jakub Jelinek wrote:
> On Thu, Jul 30, 2026 at 09:04:59PM -0400, Jason Merrill wrote:
>>> Here is an updated patch, which
>>> 1) has slightly different wording mostly as you suggested,
>>>      I have just used the pointed or referenced type rather than a pointer
>>>      (what would be printed in the reference case anyway?)
>>> 2) in cp_fold_convert I've stopped using fold_convert for the
>>>      INDIRECT_TYPE_P to INDIRECT_TYPE_P casts so that we avoid the
>>>      Convert (T1)(X p+ Y) into ((T1)X p+ Y), for pointer type, when ...
>>>      problematic fold-const.cc optimization, instead it optimizes just
>>>      some simple cases
>>> 3) had to move -fsanitize=null checking for references on nullptr in order
>>>      not to regress one ubsan testcase
>>>
>>> Bootstrapped successfull on both x86_64-linux and i686-linux, but has one
>>> regression on both:
>>> FAIL: std/ranges/adaptors/slide/1.cc  -std=gnu++23 (test for excess errors)
>>> Excess errors:
>>> /home/jakub/src/gcc/libstdc++-v3/testsuite/std/ranges/adaptors/slide/1.cc:110: error: non-constant condition for static assertion
>>> /home/jakub/src/gcc/obj74/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/ranges_base.h:1016: error: '(((((int*)(& x)) + 12) - (((int*)(& x)) + 4)) / 4)' is not a constant expression
>>> and ditto with -std=gnu++29.  I'm afraid something relies on
>>> some of the fold_convert optimizations, dunno if we should try harder
>>> during POINTER_DIFF_EXPR folding or what.
>> What if we disable the fold_unary_loc transformation if the pointers are to
>> different RECORD_TYPEs?  There are already a couple other exceptions.
> 
> Note, with the r17-2979 commit in, the patch as posted doesn't cause
> regressions.  It can be also narrowed, so in cvt.cc do fold_convert
> also if not
> +	  && CLASS_TYPE_P (TREE_TYPE (type))
> +	  && CLASS_TYPE_P (TREE_TYPE (TREE_TYPE (expr)))
> (but TYPE_MAIN_VARIANT check in that case doesn't really work,
> we have there cases we don't want to optimize where
> TREE_TYPE (type) and TREE_TYPE (TREE_TYPE (expr)) is the same type,
> but after stripping nops from expr we end up with some problematic type.
> 
> Regarding the suggested fold-const.cc change, I thought
> "Convert (T1)(X p+ Y) into ((T1)X p+ Y), for pointer type, when the new"
> is a pretty important optimization that allows moving around
> POINTER_PLUS_EXPRs so that they can be optimized together.
> If you mean
> --- gcc/fold-const.cc.jj	2026-08-04 17:00:02.424807528 +0200
> +++ gcc/fold-const.cc	2026-08-05 14:53:17.951727844 +0200
> @@ -9429,6 +9429,14 @@ fold_unary_loc (location_t loc, enum tre
>   	      && TREE_CODE (TREE_TYPE (arg00)) != REFERENCE_TYPE)
>   	    return NULL_TREE;
>   
> +	  if (RECORD_OR_UNION_TYPE_P (TREE_TYPE (type))
> +	      && RECORD_OR_UNION_TYPE_P (TREE_TYPE (TREE_TYPE (arg0)))
> +	      && (TYPE_MAIN_VARIANT (TREE_TYPE (type))
> +		  != TYPE_MAIN_VARIANT (TREE_TYPE (TREE_TYPE (arg0))))
> +	      && !in_gimple_form
> +	      && lang_GNU_CXX ())
> +	    return NULL_TREE;
> +
>   	  arg00 = fold_convert_loc (loc, type, arg00);
>   	  return fold_build_pointer_plus_loc (loc, arg00, arg01);
>   	}
> with a comment, then the disadvantage is that it will not optimize even when
> doing cp_fold* or during cp_gimplify_expr.

Yes, that's what I was thinking about.

> That said, I've been wondering if such change would prevent optimization of
> say:
> struct A { int a; };
> #ifdef __cplusplus
> struct B : A { int b; };
> struct C : B { int c; };
> #else
> struct C { int c; };
> #endif
> 
> A *
> foo (A *p)
> {
>    return ((A *) (((C *) ((A *) (((C *) ((A *) (((C *) ((A *) (((C *) p) + 1))) + 1))) + 1))) + 1));
> }
> Turns out it doesn't prevent that, because match.pd has also
> /* Associate (p +p off1) +p off2 as (p +p (off1 + off2)).  */
> (simplify
>    (pointer_plus (pointer_plus:s @0 @1) @3)
>    (pointer_plus @0 (plus @1 @3)))
> #if GENERIC
> (simplify
>    (pointer_plus (convert:s (pointer_plus:s @0 @1)) @3)
>    (pointer_plus (convert:type @0) (plus @1 @3)))
> #endif
> optimization where the GENERIC only simplification clearly
> looks also inapproprite for C++ constant evaluation.  But surprisingly
> commenting that out doesn't help, the first one triggers there too,
> so wonder what else gets rid of the casts in there.
Another possible approach to this issue would be for 
cxx_fold_pointer_plus_expression to handle the case of a NOP_EXPR on the 
lhs specially so it isn't exposed to the usual NOP_EXPR handling?

Jason