Re: Breakage in ktls-utils with nfs keyring?

On Fri, May 1, 2026, at 4:19 PM, Scott Mayhew wrote:
> On Thu, 30 Apr 2026, Chuck Lever wrote:
>
>> Cc'ing the ktls-utils development list.
>> 
>> On Thu, Apr 30, 2026, at 9:32 AM, Sagi Grimberg wrote:
>> > Hey Chuck,
>> >
>> > Upstream ktls-utils fails passing client certificate and private key 
>> > using the .nfs keyring.
>> > Bisecting leads commit facd084e43fc ("tlshd: Client-side dual 
>> > certificate support").
>> >
>> > I manually apply this (probably wrong) change and keyring works:
>> > --
>> > diff --git a/src/tlshd/client.c b/src/tlshd/client.c
>> > index 2664ffb..a946797 100644
>> > --- a/src/tlshd/client.c
>> > +++ b/src/tlshd/client.c
>> > @@ -327,7 +327,7 @@ tlshd_x509_retrieve_key_cb(gnutls_session_t session,
>> >          } else {
>> >                  tlshd_log_debug("%s: Selecting x509.certificate from 
>> > conf file", __func__);
>> >                  *pcert_length = tlshd_certs_len;
>> > -               *pcert = tlshd_certs + tlshd_pq_certs_len;
>> > +               *pcert = tlshd_certs;
>> >                  *privkey = tlshd_privkey;
>> >          }
>> >          return 0;
>> > --
>> >
>> > But, I have a feeling its not the correct change...
>> 
>> 
>> Scott, can you triage this?
>
> So when I added the dual certificate support, I didn't touch any of the
> keyring code.  Frankly, I'm not entirely sure what is the right way to
> set it up and the docs are pretty much nonexistent.  As far as I can
> tell:
>
> - you need to load nfs.ko first so that the .nfs keyring gets created
>   via nfs_init_keyring()
> - you need to restart tlshd so that it links the .nfs keyring into its
>   session keyring (I tried loading nfs.ko at boot via modules-load.d,
>   but tlshd still reported an error saying it couldn't find the .nfs
>   keyring)
> - you need to convert the cert and key to DER format
> - you need to add the cert and key to the .nfs keyring, e.g.
>
>   keyctl padd user "nfs_cert" %:.nfs < smayhew-rawhide.crt.der
>   keyctl padd user "nfs_key" %:.nfs < smayhew-rawhide.key.der
>
> - then you mount w/ '-o xprtsec=mtls,cert_serial=...,privkey_serial=...'
>
> Is that somewhat accurate?  Is there a better way to do it?  It seems
> like a lot more work than just using the config file.

It is more work because keyring support for the NFS consumers is still
aspirational/experimental.

I've pushed your patch to a "fixes" branch for folks to try out. I'm not
sure yet whether we want a 1.4.1 release with this fix, since keyring
support for NFS is "not finished".


> At any rate, I was able to reproduce the reported bug and the patch I
> just sent fixes it, but I think we probably want to make dual
> certificate support work with keyrings too.

Yes, and clearly there are some questions to be answered.


-- 
Chuck Lever