[SPDK] Consolidation of accelerator support

Hi Everyone,

As many know we have a few different areas where we support HW accelerators:


  *   The accelerator framework (fairly new, accel_fw, previously known as the copy framework)
     *   Supports generic interface for DMA and other data manipulation functions for IOAT and the upcoming DSA (aka IDXD)
     *   Supports using optimized SW calls for those API not supported by the selected hardware
  *   The compression and encryption virtual bdev modules
     *   Supports DPDK CompressDev and CryptoDev APIs
     *   Able to plug in different PMDs (polled mode drivers) as supported by SPDK, today that includes:
        *   Compression: QAT and mlx5_pci (HW) and ISAL (SW)
        *   Crypto: QAT (HW) and ISAL (SW). Ciphers: AES_CBC or AES_XTS (QAT only)

There’s been more than one discussion over the last several months about consolidating this support under one framework, Jim has asked me to work on this for the next release so I’ll be working on (a) adding all accelerator support to the accel_fw and (b) reworking the above bdev modules to use new accel_fw APIs.  This should result in a cleaner/extensible architecture and a simpler more usable set of capabilities (crypto/compression and future stuff won’t be limited to the bdev layer).

There are, of course, some challenges in coming up with the new design. In the next few weeks (I’m out next week) I’ll add an agenda item to the Euro/Americas friendly community meeting and would encourage all those interested to attend.

I haven’t put a tremendous amount of thought into this yet but for starters the accel_engine is built to have one and only one initialized HW module that drops to SW if the selected HW doesn’t support the API.  Clearly we want to be able to have the accel_fw be initialized to do either IOAT or DSA and crypto and compression, not just one of these things.  There’s a few ways to tackle this, I’ll write up a proposal prior to the community meeting in 2 weeks to discuss. Similarly, with IOAT and DSA when we don’t have HW support we use ISA-L directly however with crypto and compression you make a decision at init time whether you want HW or SW and either way you use the DPDK API just with a different PMD. The advantage here is a common API for SPDK to use regardless of the PMD (however the init sequence is slightly different depending on the PMD) but the downside is that we’re using the DPDK API for ISAL SW support when we could code that ourselves directly; this has bitten us 2-3x so far with bugs on the DPDK side so if we skip using the ISAL PMD we have a lot more control.  I don’t have an opinion on which is better yet, just something that comes to mind to think about.

If you have some short thoughts over the next few weeks please share on Slack, or here. Slack is preferred but this is a long message so I went with email ☺

Thanks!
Paul