Re: mutual authentication fails when FIPS enabled

----- Original Message -----
> From: "George" <[email protected]>
> To: [email protected]
> Sent: Wednesday, February 3, 2021 4:30:31 AM
> Subject: Re: [Opensc-devel] mutual authentication fails when FIPS enabled
> 
> I just realized that the LIBP11 function RSA_public_encrypt(...) ends up
> using a function from the OpenSSL FIPS 2.0.16 module to do the RSA
> encryption. Does that mean I can safely configure RSA_FLAG_FIPS_METHOD to be
> "true" in
> 
> 
> 
> SA_METHOD *PKCS11_get_rsa_method(void)
> {
> . . .
> RSA_meth_set_flags(ops, RSA_FLAG_FIPS_METHOD );
> . . .
> }
> ?

For me, setting this flag will tell OpenSSL that the method obtained from the
engine will perform the crypto operations in an approved module.

It is more related with the PKCS#11 module you are using than with the OpenSSL
binary you are using.

To be FIPS compliant, both OpenSSL and the PKCS#11 module you are using have
to be approved. But I can't say that is sufficient for FIPS compliance of your
whole system.

> 
> 
> I traced the code for
> 
> 
> int RSA_public_encrypt(int flen, const unsigned char *from, unsigned char
> *to, RSA *rsa, int padding)
> {
> . . .
> return (rsa->meth->rsa_pub_enc(flen, from, to, rsa, padding));
> } down to the code in openssl-fips-2.0.16\crypto\rsa\rsa_eay.c :
> 
> 
> static int RSA_eay_public_encrypt(int flen, const unsigned char *from,
> unsigned char *to, RSA *rsa, int padding)
> Since we know this if FIPS certified, does this imply the RSA encryption
> function configured in PKCS11_get_rsa_method(void) is also FIPS compliant?

I would say it is not sufficient as some crypto operations would be performed
outside the crypto boundary considered for the validation of this OpenSSL binary.

What I mean is that the OpenSSL binary is validated only for operations performed
inside it. Since you are using an external device to perform cryptographic
operations, you are not using the module in an approved way.

Best regards,
Anderson Sasaki