cms signing error with SHA3-512 & PSS

Hello list,

with the lastest openSSL 3.6.0 version, I try to sign a file with the following command (the requirements are to support PSS and SHA3-512!):

openssl cms -sign -in /tmp/input -out /tmp/output -signer /tmp/signed_key_priv.cer -keyopt rsa_padding_mode:pss -nocerts -nodetach -binary -outform DER -md sha3-512

I get the following error:

Error finalizing CMS structure
00000000:error:1C8C0103:Provider routines:rsa_generate_signature_aid:internal error:providers/implementations/signature/rsa_sig.c:352:
00000000:error:17000067:CMS routines:CMS_final:cms datafinal error:crypto/cms/cms_smime.c:926:


When changing SHA3-512 to any other SHA3 algorithm (like SHA3-256 or SHA3-384), everything seems to work fine. Also, when removing the PSS option, the command succeeds with SHA3-512. Is there anything I can improve? May using the C library with an own library call succeed? Am I missing a crucial parameter for this combination?


Regards,
Harald Koch



-- 
You received this message because you are subscribed to the Google Groups "openssl-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openssl-users+unsubscribe-MCmKBN63+Bmbup2nOX2J7Q@public.gmane.org
To view this discussion visit https://groups.google.com/a/openssl.org/d/msgid/openssl-users/C57D8068-C17D-4B5F-B2C9-0A9E937EDA00%40c-works.net.