Bug: digest parameter is rejected

RSA-OAEP supports different hash functions and MGF. SHA-1 is the default.

 

OpenSSL implementation of OAEP wrongly refuses to set the hash algorithm, preventing one from using SHA-2 family:

 

$ openssl version

OpenSSL 1.0.2l  25 May 2017

$ openssl pkeyutl -encrypt -in t1264.dat -out t1264.dat.enc2.oaep -keyform DER -pubin -inkey rsa3072pub.der -pkeyopt rsa_padding_mode:oaep

$ openssl pkeyutl -encrypt -in t1264.dat -out t1264.dat.enc2.oaep -keyform DER -pubin -inkey rsa3072pub.der -pkeyopt rsa_padding_mode:oaep -pkeyopt digest:sha256

parameter setting error

140736155067400:error:06089094:digital envelope routines:EVP_PKEY_CTX_ctrl:invalid operation:pmeth_lib.c:376:

$ ~/openssl-1.1/bin/openssl version

OpenSSL 1.1.0g-dev  xx XXX xxxx

$ ~/openssl-1.1/bin/openssl pkeyutl -encrypt -in t1264.dat -out t1264.dat.enc2.oaep -keyform DER -pubin -inkey rsa3072pub.der -pkeyopt rsa_padding_mode:oaep

$ ~/openssl-1.1/bin/openssl pkeyutl -encrypt -in t1264.dat -out t1264.dat.enc2.oaep -keyform DER -pubin -inkey rsa3072pub.der -pkeyopt rsa_padding_mode:oaep -pkeyopt digest:sha256

pkeyutl: Can't set parameter:

140736155067328:error:06089094:digital envelope routines:EVP_PKEY_CTX_ctrl:invalid operation:crypto/evp/pmeth_lib.c:312:

$

 

It seems that OpenSSL tries to enforce the incorrect assumption that digest/hash is only applicable to signature padding, but not to encryption padding?

--

Regards,

Uri Blumenthal

-- 
openssl-dev mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev