AVX-512 for LAME: code to try, if you have the hardware

Hi,

There is demand for AVX-512 optimizations. The AVX2 work in 4.1 answered 
half of that; this is an attempt at the other half, and it needs people 
with the right hardware, because we do not have any.

There is a test kit at
     https://wiki.Leidinger.net/lame-avx512-testkit.tar.gz (~870kb)
It patches trunk r6662 and adds an AVX-512 tier to the vector ladder 
that already picks between scalar, SSE2 and AVX2 at run time. Two 
kernels in the first stage - the Huffman table search and the xr^(3/4) 
quantization loop - and the VBR band-noise routine in a second.

The question is only this:

     Is it actually faster, on real hardware, end to end?

We cannot answer it. No machine here has AVX-512, and we did not use an 
emulator, so - and this is the part worth reading twice -

     THIS CODE HAS NEVER BEEN EXECUTED. NOT ONCE, ANYWHERE.

It has been compiled by four compilers, disassembled and read 
instruction by instruction, checked against the scalar code it replaces, 
and wired into the unit test suite. But the first time these 
instructions actually run on a processor will be on someone else's 
machine. A crash or a failing test is a useful result and we would 
rather hear about it than not; please do not run it on anything you care 
about.

Relevant hardware is AMD Zen 4 or Zen 5, or Intel Xeon from Skylake-SP 
onward. Current Intel desktop parts have no AVX-512 at all, and will not 
until Nova Lake. If you have one of the above and twenty minutes, the 
README in the tarball walks through it in stages; the first stage alone 
is a useful answer, and "no difference on Zen 5" is as useful as a 
speedup.

What is already done, so nobody repeats it: builds clean with 0 warnings 
on gcc 13, clang 18, clang 21 and MSVC 19.51, on Linux, FreeBSD and 
Windows; applies to a fresh r6662 checkout and builds with autotools 
removed from PATH (the regenerated build files ship in the kit, so 
autoconf/automake are not needed); and on a machine without AVX-512 the 
output is byte-identical to trunk across 519 encoder option sets, so the 
new rung does not disturb the ones below it.

Two things about how it is built:

- It is compiled on alpha versions only, and leaves the build by itself 
when
   the version type moves on. There is no configure switch, deliberately: 
a
   tier you have to opt into is a tier nobody measures. Within an alpha 
build
   it behaves like the others - what the compiler can build is built, the 
CPU
   decides at run time, and --noasm avx512 turns it off.

- The first stage cannot change the encoded output, and the kit checks 
that.
   The second stage is more interesting: the band-noise routine ends in a 
sum
   of squares, and summing sixteen values at once means summing them in a
   different order, which changes the last bit and therefore the 
bitstream.
   So it ships in two forms - one that keeps the existing order and is
   bit-exact, and one that does not and is faster. How much faster is 
exactly
   what we do not know, and it decides whether the question is worth 
having.

Nothing here is committed. If the numbers say it is not worth it, that 
is a result and the tier does not go in.

Please report back: CPU model, compiler and version, OS, the two 
timings, and anything that broke.

Bye,
Alexander.

-- 
http://www.Leidinger.net [email protected]: PGP 0x8F31830F9F2772BF
http://www.FreeBSD.org    [email protected]  : PGP 0x8F31830F9F2772BF