Re: Breakage in ktls-utils with nfs keyring?

On Thu, 30 Apr 2026, Chuck Lever wrote:

> Cc'ing the ktls-utils development list.
> 
> On Thu, Apr 30, 2026, at 9:32 AM, Sagi Grimberg wrote:
> > Hey Chuck,
> >
> > Upstream ktls-utils fails passing client certificate and private key 
> > using the .nfs keyring.
> > Bisecting leads commit facd084e43fc ("tlshd: Client-side dual 
> > certificate support").
> >
> > I manually apply this (probably wrong) change and keyring works:
> > --
> > diff --git a/src/tlshd/client.c b/src/tlshd/client.c
> > index 2664ffb..a946797 100644
> > --- a/src/tlshd/client.c
> > +++ b/src/tlshd/client.c
> > @@ -327,7 +327,7 @@ tlshd_x509_retrieve_key_cb(gnutls_session_t session,
> >          } else {
> >                  tlshd_log_debug("%s: Selecting x509.certificate from 
> > conf file", __func__);
> >                  *pcert_length = tlshd_certs_len;
> > -               *pcert = tlshd_certs + tlshd_pq_certs_len;
> > +               *pcert = tlshd_certs;
> >                  *privkey = tlshd_privkey;
> >          }
> >          return 0;
> > --
> >
> > But, I have a feeling its not the correct change...
> 
> 
> Scott, can you triage this?

So when I added the dual certificate support, I didn't touch any of the
keyring code.  Frankly, I'm not entirely sure what is the right way to
set it up and the docs are pretty much nonexistent.  As far as I can
tell:

- you need to load nfs.ko first so that the .nfs keyring gets created
  via nfs_init_keyring()
- you need to restart tlshd so that it links the .nfs keyring into its
  session keyring (I tried loading nfs.ko at boot via modules-load.d,
  but tlshd still reported an error saying it couldn't find the .nfs
  keyring)
- you need to convert the cert and key to DER format
- you need to add the cert and key to the .nfs keyring, e.g.

  keyctl padd user "nfs_cert" %:.nfs < smayhew-rawhide.crt.der
  keyctl padd user "nfs_key" %:.nfs < smayhew-rawhide.key.der

- then you mount w/ '-o xprtsec=mtls,cert_serial=...,privkey_serial=...'

Is that somewhat accurate?  Is there a better way to do it?  It seems
like a lot more work than just using the config file.

At any rate, I was able to reproduce the reported bug and the patch I
just sent fixes it, but I think we probably want to make dual
certificate support work with keyrings too.  What's the right way to go
about that?  Add PQ cert and PQ key parameters to the upcall?  Or add
lists of both PQ and RSA certs and private keys to the existing keys
and teach tlshd to parse both out of the existing keys (which I'm not
sure is possible)?

Also, is nfsd supposed to work with keyrings?  I see that tlshd looks
for a .nfsd keyring, but svc_tcp_handshake() doesn't populate ta_my_cert
and ta_my_privkey...

-Scott
> 
> 
> -- 
> Chuck Lever
>