[Bug tree-optimization/126820] New: Different behavior at -O0 and -O2/-O3 with __builtin_calloc

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=126820

            Bug ID: 126820
           Summary: Different behavior at -O0 and -O2/-O3 with
                    __builtin_calloc
           Product: gcc
           Version: 17.0
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: tree-optimization
          Assignee: unassigned at gcc dot gnu.org
          Reporter: stuzyf at 163 dot com
  Target Milestone: ---
            Target: x86-64

The following testcase behaves differently depending on the optimization level:
Reproducer: https://godbolt.org/z/c6Yv5bPMW

```
#include <stdlib.h>

int fun(int x) { volatile int y = x; return y; }

int main ()
{
  if (__builtin_calloc ((~(__SIZE_TYPE__) 0) / 16, fun(64)) )
    __builtin_abort ();
}
```

On x86-64 with GCC trunk, compiling with -O2 or -O3 produces a program that
terminates with SIGSEGV:

```
ASM generation compiler returned: 0
Execution build compiler returned: 0
Program returned: 139
Program terminated with signal: SIGSEGV
```

Without optimization, the program returns normally:

```
ASM generation compiler returned: 0
Execution build compiler returned: 0
Program returned: 0
```

This issue is reproducible with GCC 15.1 and later, but not with the older GCC
versions I tested.