Re: Bug#1121177: openblas breaks multiple autopkgtest on ppc64el

On 2026-01-07 02:27, Sébastien Villemot wrote:
> Dear Trupti,
> 
> Le mercredi 07 janvier 2026 à 02:19 +0530, Trupti a écrit :
>> On 2026-01-05 19:36, Trupti wrote:
>> > On 2026-01-02 11:53, Trupti wrote:
>> > > On 2026-01-02 00:34, Paul Gevers wrote:
>> > > > user [email protected]
>> > > > usertag 1121177 ppc64el
>> > > > thanks
>> > > >
>> > > > Dear ppc64el porters,
>> > > >
>> > > > On 11/25/25 15:41, Sébastien Villemot wrote:
>> > > >
>> > > > > I talked to upstream about the problem (in an issue that was
>> > > > > initially
>> > > > > about a FTBFS, due to a failure in OpenBLAS own testsuite, which has
>> > > > > since been fixed):
>> > > > > https://github.com/OpenMathLib/OpenBLAS/issues/5372#issuecomment-3353517450
>> > > > >
>> > > > > Unfortunately upstream does not really know where the test failures
>> > > > > in
>> > > > > third-party software come from. In particular, they can’t replicate
>> > > > > the
>> > > > > issue (note that they tried with more recent git snapshot than
>> > > > > version
>> > > > > 0.3.30), and I couldn’t either with Debian version 0.3.30+ds-3
>> > > > > (tried
>> > > > > on the ppc64el Debian porterbox).
>> > > > >
>> > > > > At this point, fixing this issue is beyond my time budget and skills
>> > > > > (I
>> > > > > know next to zero about PowerPC, and the issue is probably due to
>> > > > > some
>> > > > > changes to PowerPC assembly code). CC’ing the Debian PowerPC
>> > > > > porters,
>> > > > > with the hope that they can help.
>> > > >
>> > > >
>> > > > We're in dire need of your help, the issue is stalling openblas'
>> > > > migration to testing and because it's a key package, autoremoval
>> > > > doesn't work.
>> > > >
>> > > > Paul
>> > >
>> > >
>> > >
>> > > Thanks for the ping.
>> > >
>> > > I’m currently reproducing the issue on the ppc64el side and
>> > > investigating the root cause. Since openblas is a key package, this
>> > > needs a proper fix rather than a workaround.
>> > >
>> > > Let me go through the bug and I’ll update with findings.
>> > >
>> > > Thanks,
>> > > Trupti
>> 
>> 
>> Hello Paul,
>> 
>> I was able to reproduce the autopkgtest failure for xtensor-blas on
>> ppc64el locally. And I have attached both falling and working logs.
>> 
>> 
>> 
>> [ RUN      ] xlinalg.pinv
>> /tmp/autopkgtest.nrAywe/autopkgtest_tmp/test_linalg.cpp:239: Failure
>> Value of: allclose(expected, res)
>>    Actual: false
>> Expected: true
>> [  FAILED  ] xlinalg.pinv (0 ms)
>> 
>> [----------] Global test environment tear-down
>> [==========] 77 tests from 6 test suites ran. (7 ms total)
>> [  PASSED  ] 76 tests.
>> [  FAILED  ] 1 test, listed below:
>> [  FAILED  ] xlinalg.pinv
>> 
>>   1 FAILED TEST
>> make[3]: *** [CMakeFiles/xtest.dir/build.make:70: CMakeFiles/xtest]
>> Error 1
>> make[2]: *** [CMakeFiles/Makefile2:188: CMakeFiles/xtest.dir/all] 
>> Error
>> 2
>> make[1]: *** [CMakeFiles/Makefile2:195: CMakeFiles/xtest.dir/rule] 
>> Error
>> 2
>> make: *** [Makefile:192: xtest] Error 2
>> autopkgtest [23:35:52]: test command2: -----------------------]
>> autopkgtest [23:35:52]: test command2:  - - - - - - - - - - results - 
>> -
>> - - - - - - - -
>> command2             FAIL non-zero exit status 2
>> autopkgtest [23:35:52]: @@@@@@@@@@@@@@@@@@@@ summary
>> command1             FAIL non-zero exit status 2
>> command2             FAIL non-zero exit status 2
>> 
>> 
>> The failure occurs in the test:
>> xlinalg.pinv
>> test/test_linalg.cpp
>> 
>> 
>> When running the test locally on ppc64el with OpenBLAS 0.3.30, the
>> maximum numerical difference between the expected result and
>> xt::linalg::pinv() output is:
>> 
>> max diff ≈ 7.0e-09
>> mean diff ≈ 2.7e-09
>> 
>> 
>> With the current test tolerance (allclose default / 1e-12), the test
>> fails.
>> When the tolerance is relaxed to 1e-8, the test passes consistently 
>> and
>> all results are numerically stable.
>> 
>> This indicates the failure is due to test tolerance rather than a
>> functional regression.
>> kindly consider reviewing the test tolerance.
> 
> Thanks a lot for your investigation and for the recommendation.
> 
> If you have the time, could you possibly also check that the two other
> autopkgtest regressions (in src:gemma and src:openmolcas) are also
> tolerance-related? (see https://tracker.debian.org/pkg/openblas for the
> list of autopkgtest regressions)


Yes, I will do it. And share you the results as soon as possible.


Thanks,
Trupti