Re: mutual authentication fails when FIPS enabled

Hi,

    I had a closer look at this problem and confirmed that I have 
successfully enabled FIPS in OpenSSL and am using the FIPS compliant 
cipher suite TLS_RSA_WITH_AES_128_CBC_SHA. 
(https://wiki.openssl.org/index.php/FIPS_mode_and_TLS#TLS_1.2)

However, LIBP11 still fails unless I make a modification to the source 
code in p11_rsa.c. Does this mean there is indeed a bug in LIBP11?

I changed the line
RSA_meth_set_flags(ops, 0);
to
RSA_meth_set_flags(ops, *RSA_FLAG_FIPS_METHOD*);

in the code

    RSA_METHOD *PKCS11_get_rsa_method(void)
    {
         static RSA_METHOD *ops = NULL;

         if (!ops) {
             alloc_rsa_ex_index();
             ops = RSA_meth_dup(RSA_get_default_method());
             if (!ops)
                 return NULL;
             RSA_meth_set1_name(ops, "libp11 RSA method");
             RSA_meth_set_flags(ops, RSA_FLAG_FIPS_METHOD);
             RSA_meth_set_priv_enc(ops, pkcs11_rsa_priv_enc_method);
             RSA_meth_set_priv_dec(ops, pkcs11_rsa_priv_dec_method);
             RSA_meth_set_finish(ops, pkcs11_rsa_free_method);
         }
         return ops;
    }

(This was desribed in https://bugzilla.redhat.com/show_bug.cgi?id=1827535 )

The flag RSA_FLAG_FIPS_METHOD is defined to specify the RSA methods are 
FIPS compliant:

    /* If this flag is set the RSA method is FIPS compliant and can be used
    * in FIPS mode. This is set in the validated module method. If an
    * application sets this flag in its own methods it is its responsibility
    * to ensure the result is compliant.
    */
    #define RSA_FLAG_FIPS_METHOD            0x0400



Thanks,
George


On 2021-01-28 2:03 a.m., George wrote:
> Thanks for the information. Do you mean the line
> RSA_meth_set_flags(ops, RSA_FLAG_FIPS_METHOD);
> will allow the inclusion of non-FIPS compliant algorithms?
>
>  Does this mean I should not be using LIBP11 if I want my code to be 
> FIPS-compliant?
>
>
> Thanks,
> George
>
>
> On 2021-01-27 2:54 a.m., Petr Pisar wrote:
>> V Tue, Jan 26, 2021 at 11:25:13PM -0500, George napsal(a):
>>>    When I attempt to do mutual authentication with a smart card, it
>>> fails. The problem appears to be related to enabling FIPS. I am using
>>> OpenSSL 1.0.2u(with FIPS) and LIBP11 0.4.11.
>>> The found the following Red Hat bug report, which describes the exact
>>> same problem I am seeing:
>>> https://bugzilla.redhat.com/show_bug.cgi?id=1827535
>>>
>>> Is this a known LIBP11 bug? Will it be fixed in future versions of LIBP11?
>>>
>>> A suggested workaround in the above link is to make a change in the file
>>> libp11-libp11-0.4.11\src\p11_rsa.c:
>>> Change the line
>>>
>>>      RSA_meth_set_flags(ops, 0);
>>>
>>> to
>>>
>>>      RSA_meth_set_flags(ops, RSA_FLAG_FIPS_METHOD);
>>>
>>>
>>> Once I did this, it fixed my problem. Is this a proper fix?
>>>
>> The flag disables exclusion of FIPS noncompliant algorithms (e.g. MD5) from
>> the OpenSSL routines and leaves the compliance to the application.
>>
>> In other words your system is not FIPS compliant anymore unless you get FIPS
>> certification for your patched libp11.
>>
>> The questions are:
>>
>> Do you really need FIPS compliance? If not, then do not enable FIPS mode.
>>
>> Does the mutual authentication protocol manadate use of noncompliant
>> algorithms? If yes, then it's broken by desisign and you simply cannot use the
>> authentication in FIPS mode. Otherwise libp11 and your smart card should use
>> a different, compliant algorithm instead.
>>
>> Is smart card mutual authentication subject of FIPS compliance? Technically
>> you can use non-compliant algorithms for non-cryprographical purposes even in
>> FIPS mode. E.g. MD5 for hash tables or data identification. But in my opinion
>> authentication falls into a FIPS realm, so this not the case. You should talk
>> to your security auditor about this.
>>
>> -- Petr
>>
>>
>> _______________________________________________
>> Opensc-devel mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/opensc-devel
>

_______________________________________________
Opensc-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/opensc-devel