Re: [PATCH v4 3/3] tee: add MbedTEE driver
Jens Wiklander via OP-TEE <[email protected]>
| Newsgroups | org.trustedfirmware.lists.op-tee,org.kernel.vger.linux-devicetree,org.kernel.vger.linux-kernel |
|---|---|
| Message-ID | <CAGgiveU2TjkA1YnPxBy2e9qtCorSumuno04aEir-u74yKvXBsw@mail.gmail.com> |
Hi, On Wed, Aug 19, 2026 at 10:37 AM Xing Loong <[email protected]> wrote: > > This patch adds a Linux TEE driver that communicates with > MbedTEE OS, a Trusted Execution Environment for embedded > systems (https://github.com/mbedtee) via a fixed shared-memory > RPC protocol over architecture-specific transports: > > - ARM/ARM64: SMCCC fast/yield calls for REE-to-TEE requests; > TEE-to-REE notifications via a GIC SPI edge interrupt. > - RISC-V: REE submits commands by writing to a shared-memory > r2t-ring that the TEE polls; the TEE notifies the REE by > writing to t2r-ring and raising a platform IMSIC MSI. > No REE-to-TEE interrupt is used. > > The driver implements the TEE subsystem interface (tee_driver_ops) > and provides GlobalPlatform TEE Client API support, dynamic shared > memory registration, and tee-supplicant support. > > Signed-off-by: Xing Loong <[email protected]> I took a quick look at the mbedtee project, and it appears to be a one-person hobby project. Sounds like fun, but I wonder how meaningful it is to upstream a kernel driver if you're the only user. Cheers, Jens > --- > Changes in v4: > - Harden ring and RPC handling against malformed input: > - Validate both ring indices; bound async ring skip and resync > the reader on corrupt framing > - Enforce RPC handler modes (COMPLETE_REE async-only, supplicant > commands sync-only) > - Reference-count rpc_call objects; fix use-after-free on > interrupted calls; abort and free leftover calls at teardown > - Add XA_FLAGS_LOCK_IRQ to the rpc_calls xarray > - Fix supp_recv/supp_abort_all race; malformed supp_send unblocks > the worker instead of hanging it > - Reject TEE-changed parameter attributes > - Report the wire error on failed supplicant requests; fail > oversized requests instead of requeueing them forever > - Wait for R2T ring space with a bounded backoff; bound > COMPLETE_TEE retries and stop them at teardown > - Match the supplicant context when dispatching supp_send > - Access complete_work_pending under ring_lock > - Serialize RPC submission against shutdown and reject new > calls once teardown starts > - Quiesce all RPC activity before tee_device_unregister() so > parked waiters cannot deadlock teardown > - Bound non-interruptible RISC-V waits so a hung TEE cannot > block ioctls forever > - Track closing state on sessions; reject concurrent > close/invoke/cancel races > - Quarantine TEE-owned call objects at release instead of > risking a cross-world use-after-free > > Changes in v3: > - No changes (driver unchanged since v2) > > Changes in v2: > - Fix supp_release incorrectly aborting unclaimed requests on close > - Fix potential tee_shm double-free on supp_recv error path > - Fix async RPC ring skip leaving orphaned payload bytes > - Fix COMPLETE_TEE retry to also handle transient -ENOMEM on RISC-V > - Fix session leak on close_session allocation failure: > release kernel resources before sending RPC > > --- > Documentation/tee/index.rst | 1 + > Documentation/tee/mbedtee.rst | 155 ++++++ > MAINTAINERS | 9 + > drivers/tee/Kconfig | 3 +- > drivers/tee/Makefile | 1 + > drivers/tee/mbedtee/Kconfig | 20 + > drivers/tee/mbedtee/Makefile | 11 + > drivers/tee/mbedtee/core.c | 260 +++++++++ > drivers/tee/mbedtee/mbedtee_drv.h | 289 ++++++++++ > drivers/tee/mbedtee/mbedtee_msg.h | 219 ++++++++ > drivers/tee/mbedtee/rpc_callee.c | 697 +++++++++++++++++++++++++ > drivers/tee/mbedtee/rpc_callee_arm.c | 91 ++++ > drivers/tee/mbedtee/rpc_callee_riscv.c | 203 +++++++ > drivers/tee/mbedtee/rpc_caller.c | 697 +++++++++++++++++++++++++ > drivers/tee/mbedtee/rpc_caller_arm.c | 70 +++ > drivers/tee/mbedtee/rpc_caller_riscv.c | 238 +++++++++ > drivers/tee/mbedtee/shm_pool.c | 105 ++++ > drivers/tee/mbedtee/shm_pool.h | 15 + > drivers/tee/mbedtee/supp.c | 311 +++++++++++ > include/uapi/linux/tee.h | 1 + > 20 files changed, 3395 insertions(+), 1 deletion(-) > create mode 100644 Documentation/tee/mbedtee.rst > create mode 100644 drivers/tee/mbedtee/Kconfig > create mode 100644 drivers/tee/mbedtee/Makefile > create mode 100644 drivers/tee/mbedtee/core.c > create mode 100644 drivers/tee/mbedtee/mbedtee_drv.h > create mode 100644 drivers/tee/mbedtee/mbedtee_msg.h > create mode 100644 drivers/tee/mbedtee/rpc_callee.c > create mode 100644 drivers/tee/mbedtee/rpc_callee_arm.c > create mode 100644 drivers/tee/mbedtee/rpc_callee_riscv.c > create mode 100644 drivers/tee/mbedtee/rpc_caller.c > create mode 100644 drivers/tee/mbedtee/rpc_caller_arm.c > create mode 100644 drivers/tee/mbedtee/rpc_caller_riscv.c > create mode 100644 drivers/tee/mbedtee/shm_pool.c > create mode 100644 drivers/tee/mbedtee/shm_pool.h > create mode 100644 drivers/tee/mbedtee/supp.c > > diff --git a/Documentation/tee/index.rst b/Documentation/tee/index.rst > index 10c3cec..6415855 100644 > --- a/Documentation/tee/index.rst > +++ b/Documentation/tee/index.rst > @@ -12,3 +12,4 @@ TEE Subsystem > amd-tee > ts-tee > qtee > + mbedtee > diff --git a/Documentation/tee/mbedtee.rst b/Documentation/tee/mbedtee.rst > new file mode 100644 > index 0000000..dfb57b8 > --- /dev/null > +++ b/Documentation/tee/mbedtee.rst > @@ -0,0 +1,155 @@ > +.. SPDX-License-Identifier: GPL-2.0 > + > +============================================== > +MbedTEE (Trusted Execution Environment) > +============================================== > + > +The MbedTEE driver supports MbedTEE-based TEEs on ARM TrustZone (SMC calls, > +GIC SPI notifications) and RISC-V IMSIC (shared-memory polling, MSI > +notifications) platforms. > + > +The driver requires the REE and TEE CPUs that share the RPC shared memory to > +be hardware coherent. In practice, the REE/TEE CPUs are expected to be in the > +same CPU cluster, with coherent caches and shared visibility for the ring and > +payload buffers. > + > +Communication overview > +====================== > + > +The driver communicates with the TEE using a fixed shared-memory RPC protocol > +layered above the architecture-specific call mechanism. > + > +ARM platforms (SMC) > +------------------- > + > +On ARM and AArch64 platforms, the driver uses ARM SMC Calling Convention > +(SMCCC) to issue fast calls and yield calls to the TEE: > + > +- **Fast calls** (MBEDTEE_RPC_OS_VERSION, MBEDTEE_RPC_SUPPORT_YIELD, > + MBEDTEE_RPC_COMPLETE_TEE) are self-contained SMC calls that do > + not rely on shared-memory RPC structures. > + > +- **Yield calls** post the physical address of an ``rpc_cmd`` structure > + in shared memory to the TEE. The TEE may issue RPC requests back to the > + driver before the call completes; kernel-directed RPCs are handled > + directly, while supplicant RPCs (filesystem and RPMB) are forwarded to > + tee-supplicant. > + > +TEE-to-REE notifications on ARM are delivered via a GIC SPI edge interrupt > +specified in the device tree (``interrupts`` property). > + > +RISC-V platforms > +---------------- > + > +On RISC-V platforms, SMCCC is not available. Communication is split into two > +directions: > + > +- **REE to TEE (R2T)**: The driver submits commands by writing the physical > + address of an ``rpc_cmd`` structure to a REE-to-TEE ring buffer > + (``r2t-ring``). The TEE polls that ring for new commands. No interrupt > + notification is sent; the protocol relies on TEE-side polling. > + > +- **TEE to REE (T2R)**: The TEE writes an ``rpc_cmd`` into the TEE-to-REE > + ring and raises an IMSIC MSI to notify the REE driver. The MSI is > + allocated at runtime via the ``msi-parent`` DT property and its identity > + is published in ``callee_imsic_id``; ``callee_hartid`` tracks the > + target hart for migration during CPU hotplug. No SBI ecall is involved. > + > +RISC-V REE-to-TEE polling rationale > +----------------------------------- > + > +The REE-to-TEE direction uses polling-only on RISC-V to avoid direct > +notification interrupt writes from Linux to TEE-owned interrupt files. > +This design is platform-independent and does not require ownership of > +TEE-only hart interrupt files. The TEE-to-REE direction remains fully > +functional via standard Linux MSI notifications and does not depend on > +REE-to-TEE notification latency. > + > +Shared memory regions > +===================== > + > +Two or three fixed shared memory regions are described in the device tree: > + > +``t2r-ring`` > + Ring buffer used by the TEE to post RPC request notifications to the REE > + driver. Present on all platforms. > + > +``t2r-shm`` > + Shared memory region carrying the actual ``rpc_cmd`` payloads for > + TEE-to-REE RPCs. Present on all platforms. > + > +``r2t-ring`` > + Ring buffer used by the REE driver to submit commands to the TEE on > + RISC-V IMSIC platforms. > + > +RPC protocol > +============ > + > +The TEE and REE communicate through the ``rpc_cmd`` structure in shared memory > +and ring buffers:: > + > + struct rpc_cmd { > + u32 id; /* RPC function ID */ > + u16 size; /* payload size in bytes */ > + u8 interrupted; /* set if wait was interrupted */ > + u8 reserved; /* explicit alignment padding */ > + s32 ret; /* return value */ > + u32 pad; /* explicit alignment padding */ > + u64 waiter_id; /* sync RPC request ID echoed on completion */ > + u64 shm; /* physical address of payload (sync RPC) */ > + u64 data[]; /* inline payload (async RPC) */ > + }; > + > + struct rpc_ringbuf { > + u32 wr; /* producer write pointer */ > + u32 rd; /* consumer read pointer */ > + u32 callee_ready; /* callee ready flag */ > + u32 callee_imsic_id; /* RISC-V only: IMSIC local interrupt id */ > + u32 callee_hartid; /* RISC-V only: target hart-id for T2R notification */ > + u32 reserved; /* padding, must be zero */ > + u8 mem[]; > + }; > + > +For RISC-V T2R MSI, one MSI message targets one hart IMSIC file at a time. > +The wire-visible state is split between ``callee_imsic_id`` and > +``callee_hartid`` in the ring header; Linux may retarget the MSI across > +online CPUs via ``irq_set_affinity()``, and the driver updates those fields > +to match the selected hart. > + > + > + > +Architecture diagram:: > + > + User space Kernel TEE side > + ~~~~~~~~~~ ~~~~~~ ~~~~~~~~~~~~ > + +--------+ +--------------+ > + | Client | | Trusted App | > + +--------+ +--------------+ > + /\ /\ > + || +------------+ || > + || | MbedTEE- | \/ > + || | supplicant | +--------------+ > + || +------------+ | TEE Internal | > + \/ /\ | API | > + +-------+ || +--------------+ > + | TEE | || +---------+---------------+ | MbedTEE | > + | Client| || | TEE | MbedTEE | | Trusted OS | > + | API | \/ | subsys | client driver | +--------------+ > + +-------+-------------+-----+--------+----------+----+ | > + | Generic TEE API | | RPC (cmd/ring)| | > + | IOCTL (TEE_IOC_*) | | SMC / IMSIC | | > + +---------------------------+ +---------------+--------------+ > + > +Device tree binding > +=================== > + > +See Documentation/devicetree/bindings/firmware/mbedtee,tee.yaml for the > +complete device tree binding specification, including the RISC-V > +``msi-parent`` requirement. > + > +References > +========== > + > +- [1] MbedTEE project: https://github.com/mbedtee > +- [2] ARM SMC Calling Convention: https://developer.arm.com/architectures/system-architectures/software-standards/smccc > +- [3] RISC-V IMSIC specification: https://github.com/riscv/riscv-aia > diff --git a/MAINTAINERS b/MAINTAINERS > index c8d4b91..344b82b 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -15920,6 +15920,15 @@ F: Documentation/devicetree/bindings/net/dsa/maxlinear,mxl862xx.yaml > F: drivers/net/dsa/mxl862xx/ > F: net/dsa/tag_mxl862xx.c > > +MBEDTEE TEE DRIVER > +M: Xing Loong <[email protected]> > +L: [email protected] (moderated for non-subscribers) > +S: Maintained > +W: https://github.com/mbedtee > +F: Documentation/devicetree/bindings/firmware/mbedtee,tee.yaml > +F: Documentation/tee/mbedtee.rst > +F: drivers/tee/mbedtee/ > + > MCAN DEVICE DRIVER > M: Markus Schneider-Pargmann <[email protected]> > L: [email protected] > diff --git a/drivers/tee/Kconfig b/drivers/tee/Kconfig > index 98c3ad0..3dc3439 100644 > --- a/drivers/tee/Kconfig > +++ b/drivers/tee/Kconfig > @@ -2,7 +2,7 @@ > # Generic Trusted Execution Environment Configuration > menuconfig TEE > tristate "Trusted Execution Environment support" > - depends on HAVE_ARM_SMCCC || COMPILE_TEST || CPU_SUP_AMD > + depends on HAVE_ARM_SMCCC || RISCV || COMPILE_TEST || CPU_SUP_AMD > select CRYPTO_LIB_SHA1 > select DMA_SHARED_BUFFER > select GENERIC_ALLOCATOR > @@ -21,5 +21,6 @@ source "drivers/tee/optee/Kconfig" > source "drivers/tee/amdtee/Kconfig" > source "drivers/tee/tstee/Kconfig" > source "drivers/tee/qcomtee/Kconfig" > +source "drivers/tee/mbedtee/Kconfig" > > endif > diff --git a/drivers/tee/Makefile b/drivers/tee/Makefile > index 3239b91..33117ea 100644 > --- a/drivers/tee/Makefile > +++ b/drivers/tee/Makefile > @@ -8,3 +8,4 @@ obj-$(CONFIG_OPTEE) += optee/ > obj-$(CONFIG_AMDTEE) += amdtee/ > obj-$(CONFIG_ARM_TSTEE) += tstee/ > obj-$(CONFIG_QCOMTEE) += qcomtee/ > +obj-$(CONFIG_MBEDTEE) += mbedtee/ > diff --git a/drivers/tee/mbedtee/Kconfig b/drivers/tee/mbedtee/Kconfig > new file mode 100644 > index 0000000..b1fa38a > --- /dev/null > +++ b/drivers/tee/mbedtee/Kconfig > @@ -0,0 +1,20 @@ > +# SPDX-License-Identifier: GPL-2.0-only > +# MbedTEE Trusted Execution Environment Configuration > +config MBEDTEE > + tristate "MbedTEE" > + depends on MMU && OF > + depends on ((ARM || ARM64) && HAVE_ARM_SMCCC) || (RISCV && GENERIC_MSI_IRQ) > + help > + This enables the MbedTEE Trusted Execution Environment (TEE) > + driver. > + > + The driver supports ARM TrustZone (SMC calls, GIC SPI notifications) > + and RISC-V IMSIC (shared-memory polling, MSI notifications), with > + GlobalPlatform client API via /dev/tee*. > + > + The DT node should be under a firmware node and have compatible = > + "mbedtee,tee" with the RPC ring buffer addresses and, > + for ARM/ARM64, an interrupt property for TEE-to-REE notifications. > + See Documentation/tee/mbedtee.rst for protocol details. > + > + If unsure, say N. > diff --git a/drivers/tee/mbedtee/Makefile b/drivers/tee/mbedtee/Makefile > new file mode 100644 > index 0000000..309eb6d > --- /dev/null > +++ b/drivers/tee/mbedtee/Makefile > @@ -0,0 +1,11 @@ > +# SPDX-License-Identifier: GPL-2.0-only > + > +obj-$(CONFIG_MBEDTEE) += mbedtee.o > + > +mbedtee-y += core.o > +mbedtee-y += supp.o > +mbedtee-y += shm_pool.o > +mbedtee-y += rpc_caller.o rpc_callee.o > +mbedtee-$(CONFIG_ARM) += rpc_caller_arm.o rpc_callee_arm.o > +mbedtee-$(CONFIG_ARM64) += rpc_caller_arm.o rpc_callee_arm.o > +mbedtee-$(CONFIG_RISCV) += rpc_caller_riscv.o rpc_callee_riscv.o > diff --git a/drivers/tee/mbedtee/core.c b/drivers/tee/mbedtee/core.c > new file mode 100644 > index 0000000..4a027a7 > --- /dev/null > +++ b/drivers/tee/mbedtee/core.c > @@ -0,0 +1,260 @@ > +// SPDX-License-Identifier: GPL-2.0-only > +/* > + * Copyright (c) 2020 Xing Loong <[email protected]> > + */ > +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt > + > +#include <linux/errno.h> > +#include <linux/io.h> > +#include <linux/module.h> > +#include <linux/of.h> > +#include <linux/of_address.h> > +#include <linux/platform_device.h> > +#include <linux/slab.h> > +#include <linux/string.h> > +#include <linux/tee_drv.h> > +#include <linux/types.h> > + > +#include "mbedtee_drv.h" > +#include "shm_pool.h" > + > +int mbedtee_get_resource(struct device_node *node, > + const char *name, struct resource *res) > +{ > + struct device_node *rmem; > + int idx; > + int ret; > + > + idx = of_property_match_string(node, "memory-region-names", name); > + if (idx < 0) > + return idx; > + > + rmem = of_parse_phandle(node, "memory-region", idx); > + if (!rmem) > + return -ENODEV; > + > + ret = of_address_to_resource(rmem, 0, res); > + of_node_put(rmem); > + > + if (ret) > + return ret; > + > + res->name = name; > + return 0; > +} > + > +static void mbedtee_get_version(struct tee_device *teedev, > + struct tee_ioctl_version_data *vers) > +{ > + struct tee_ioctl_version_data v = { > + .impl_id = TEE_IMPL_ID_MBEDTEE, > + .impl_caps = 0, > + .gen_caps = TEE_GEN_CAP_GP | TEE_GEN_CAP_REG_MEM | > + TEE_GEN_CAP_MEMREF_NULL, > + }; > + > + *vers = v; > +} > + > +static int mbedtee_open(struct tee_context *ctx) > +{ > + struct mbedtee_context_data *ctxdata; > + > + ctxdata = kzalloc_obj(*ctxdata, GFP_KERNEL); > + if (!ctxdata) > + return -ENOMEM; > + > + mutex_init(&ctxdata->mutex); > + INIT_LIST_HEAD(&ctxdata->sess_list); > + > + ctx->data = ctxdata; > + ctx->cap_memref_null = true; > + > + return 0; > +} > + > +static void mbedtee_release(struct tee_context *ctx) > +{ > + struct mbedtee_device *mbedtee = tee_get_drvdata(ctx->teedev); > + struct mbedtee_context_data *d = ctx->data; > + struct mbedtee_session *s, *n; > + > + if (!d) > + return; > + > + if (d->is_supp_ctx) { > + mbedtee_supp_release(&mbedtee->supp, ctx); > + } else { > + list_for_each_entry_safe(s, n, &d->sess_list, list_node) { > + u32 session_id = s->session_id; > + > + if (!mbedtee_close_session(ctx, session_id)) > + continue; > + > + /* > + * A non-zero return can mean the session was already > + * dropped (e.g. -ESHUTDOWN during teardown) or kept on a > + * transient error. Re-look it up: kfree(NULL) is a no-op > + * in the former case, a force-removal in the latter. > + */ > + mutex_lock(&d->mutex); > + s = mbedtee_find_session_locked(d, session_id); > + if (s) > + list_del(&s->list_node); > + mutex_unlock(&d->mutex); > + kfree(s); > + } > + } > + > + kfree(d); > + ctx->data = NULL; > +} > + > +static const struct tee_driver_ops mbedtee_ops = { > + .get_version = mbedtee_get_version, > + .open = mbedtee_open, > + .release = mbedtee_release, > + .open_session = mbedtee_open_session, > + .close_session = mbedtee_close_session, > + .invoke_func = mbedtee_invoke_func, > + .cancel_req = mbedtee_cancel_req, > + .supp_recv = mbedtee_supp_recv, > + .supp_send = mbedtee_supp_send, > + .shm_register = mbedtee_shm_register, > + .shm_unregister = mbedtee_shm_unregister, > +}; > + > +static const struct tee_desc mbedtee_desc = { > + .name = "mbedtee", > + .ops = &mbedtee_ops, > + .owner = THIS_MODULE, > +}; > + > +static int mbedtee_probe(struct platform_device *pdev) > +{ > + int ret; > + long version; > + long yield; > + struct tee_shm_pool *pool; > + struct tee_device *teedev; > + struct mbedtee_device *mbedtee; > + > + mbedtee = devm_kzalloc(&pdev->dev, sizeof(*mbedtee), GFP_KERNEL); > + if (!mbedtee) > + return -ENOMEM; > + > + mbedtee->dev = &pdev->dev; > + xa_init_flags(&mbedtee->rpc_calls, XA_FLAGS_ALLOC1 | XA_FLAGS_LOCK_IRQ); > + mutex_init(&mbedtee->rpc_calls_mutex); > + mbedtee_supp_init(&mbedtee->supp); > + > + /* > + * R2T must be ready before T2R is advertised: synchronous T2R RPC > + * work replies with COMPLETE_TEE over the R2T path. > + */ > + ret = mbedtee_r2t_init(mbedtee); > + if (ret != 0) > + goto err_supp; > + > + ret = mbedtee_rpc_init(mbedtee); > + if (ret != 0) > + goto err_r2t; > + > + version = mbedtee_rpc_fastcall(mbedtee, MBEDTEE_RPC_OS_VERSION, 0, 0, 0); > + if (version < 0) { > + dev_err(&pdev->dev, "MBEDTEE_RPC_OS_VERSION failed: %ld\n", version); > + ret = version; > + goto err_rpc; > + } > + > + if (!MBEDTEE_VALID_VERSION(version)) { > + dev_err(&pdev->dev, "mbedtee not present (version=0x%lx)\n", > + version); > + ret = -ENODEV; > + goto err_rpc; > + } > + > + yield = mbedtee_rpc_fastcall(mbedtee, MBEDTEE_RPC_SUPPORT_YIELD, 0, 0, 0); > + dev_info(&pdev->dev, "version: 0x%06lx yield: %ld\n", version, yield); > + > + pool = mbedtee_shm_pool_alloc_pages(); > + if (IS_ERR(pool)) { > + ret = PTR_ERR(pool); > + goto err_rpc; > + } > + > + teedev = tee_device_alloc(&mbedtee_desc, &pdev->dev, pool, mbedtee); > + if (IS_ERR(teedev)) { > + ret = PTR_ERR(teedev); > + goto err_pool; > + } > + > + ret = tee_device_register(teedev); > + if (ret) > + goto err_teedev; > + > + mbedtee->teedev = teedev; > + mbedtee->version = version; > + mbedtee->yield = !!yield; > + mbedtee->pool = pool; > + platform_set_drvdata(pdev, mbedtee); > + > + dev_dbg(&pdev->dev, "client initialized\n"); > + return 0; > + > +err_teedev: > + tee_device_unregister(teedev); > +err_pool: > + tee_shm_pool_free(pool); > +err_rpc: > + mbedtee_rpc_uninit(mbedtee); > +err_r2t: > + mbedtee_r2t_uninit(mbedtee); > +err_supp: > + mbedtee_supp_uninit(&mbedtee->supp); > + mbedtee_rpc_calls_release(mbedtee); > + return ret; > +} > + > +static void mbedtee_remove(struct platform_device *pdev) > +{ > + struct mbedtee_device *mbedtee = platform_get_drvdata(pdev); > + > + /* > + * Quiesce and wake all RPC waiters before tee_device_unregister(), > + * which blocks until every context is released; a waiter still parked > + * in a completion would otherwise deadlock teardown. > + */ > + mbedtee_rpc_stop(mbedtee); > + tee_device_unregister(mbedtee->teedev); > + /* > + * Drain T2R before tearing down R2T: queued RPC work may still need > + * to send COMPLETE_TEE over R2T. Destroy the supplicant mutex only > + * after RPC work can no longer enter mbedtee_supp_handler(). > + */ > + mbedtee_rpc_uninit(mbedtee); > + mbedtee_r2t_uninit(mbedtee); > + mbedtee_supp_uninit(&mbedtee->supp); > + tee_shm_pool_free(mbedtee->pool); > + mbedtee_rpc_calls_release(mbedtee); > +} > + > +static const struct of_device_id mbedtee_dt_match[] = { > + { .compatible = "mbedtee,tee" }, > + { }, > +}; > +MODULE_DEVICE_TABLE(of, mbedtee_dt_match); > + > +static struct platform_driver mbedtee_driver = { > + .probe = mbedtee_probe, > + .remove = mbedtee_remove, > + .driver = { > + .name = "mbedtee", > + .of_match_table = mbedtee_dt_match, > + }, > +}; > +module_platform_driver(mbedtee_driver); > + > +MODULE_AUTHOR("Xing Loong <[email protected]>"); > +MODULE_DESCRIPTION("MbedTEE Trusted Execution Environment driver"); > +MODULE_LICENSE("GPL"); > diff --git a/drivers/tee/mbedtee/mbedtee_drv.h b/drivers/tee/mbedtee/mbedtee_drv.h > new file mode 100644 > index 0000000..fe2bcfa > --- /dev/null > +++ b/drivers/tee/mbedtee/mbedtee_drv.h > @@ -0,0 +1,289 @@ > +/* SPDX-License-Identifier: GPL-2.0-only */ > +/* > + * Copyright (c) 2020 Xing Loong <[email protected]> > + */ > +#ifndef MBEDTEE_DRV_H > +#define MBEDTEE_DRV_H > + > +#include <linux/types.h> > +#include <linux/errno.h> > +#include <linux/atomic.h> > +#include <linux/of.h> > +#include <linux/ioport.h> > +#include <linux/tee_drv.h> > +#include <linux/tee_core.h> > +#include <linux/interrupt.h> > +#include <linux/platform_device.h> > +#include <linux/msi.h> > +#include <linux/mutex.h> > +#include <linux/workqueue.h> > +#include <linux/cpuhotplug.h> > +#include <linux/cpumask.h> > +#include <linux/refcount.h> > +#include <linux/xarray.h> > +#include "mbedtee_msg.h" > +#include "shm_pool.h" > + > +/* > + * GlobalPlatform TEE Client API return codes/origins > + * used by this driver. Only the codes actually > + * referenced in the driver are defined here. > + */ > +#define TEEC_SUCCESS 0x00000000 > +#define TEEC_ERROR_OUT_OF_MEMORY 0xFFFF000C > +#define TEEC_ERROR_COMMUNICATION 0xFFFF000E > + > +#define TEEC_ORIGIN_COMMS 0x00000002 > + > +#define MBEDTEE_VERSION_MAJOR 1 > +#define MBEDTEE_VALID_VERSION(x) (((x) >> 16) == MBEDTEE_VERSION_MAJOR) > + > +/* Maximum payload size for inline (async) RPC data. */ > +#define MBEDTEE_ASYNC_RPC_DATA_MAX 256 > + > +struct rpc_transport_ops; > +struct mbedtee_rpc_call; > +struct mbedtee_supp_req; > +struct mbedtee_device; > + > +struct rpc_work { > + struct work_struct work; > + struct mbedtee_device *mbedtee; > + u64 waiter_id; > + size_t size; > + bool complete_only; > + /* Points to ctx->rpc_data (async) or into t2r_shm (sync) */ > + void *data; > + void (*func)(struct mbedtee_device *mbedtee, void *data, size_t size); > +}; > + > +struct mbedtee_t2r_ctx { > + spinlock_t ring_lock; /* protects T2R ring producer/consumer state */ > + void *t2r_shm; > + phys_addr_t t2r_shm_phys; > + resource_size_t t2r_shm_sz; > + struct rpc_ringbuf *t2r_ring; > + u32 t2r_ring_sz; > + u32 t2r_ring_rd; > + > + void (*rpc_handlers[MBEDTEE_RPC_MAX])(struct mbedtee_device *mbedtee, > + void *data, size_t size); > + > + /* > + * Inline buffer for async RPC payloads (waiter_id == 0). > + * Sync RPC payloads (waiter_id != 0) live in shared memory (t2r_shm). > + */ > + u8 rpc_data[MBEDTEE_ASYNC_RPC_DATA_MAX]; > + > + /* > + * CPU-hotplug tracking for T2R interrupt delivery. > + * callee_cpus: shadow of cpu_online_mask maintained by the core > + * callee layer (rpc_callee.c) for both ARM and RISC-V; used by > + * ARM to feed irq_set_affinity() and by RISC-V as the gate for > + * MSI migration decisions. > + * callee_virq: single MSI virq currently pinned to one CPU > + * (RISC-V IMSIC only; 0 for ARM). > + * hp_node: per-instance node for cpuhp_setup_state_multi(). > + */ > + struct cpumask callee_cpus; > + int callee_virq; > + struct hlist_node hp_node; > + bool cpuhp_added; > + bool complete_work_pending; > + bool pending_async; > + bool stopping; /* set at teardown to stop COMPLETE_TEE retries */ > + u32 pending_size; > + struct workqueue_struct *rpc_wq; > + struct rpc_work complete_work; > + void (*pending_func)(struct mbedtee_device *mbedtee, > + void *data, size_t size); > +}; > + > +struct mbedtee_r2t_ctx { > + spinlock_t lock; /* protects R2T ring write-side state */ > + struct rpc_ringbuf *ring; > + u32 ring_sz; > + u32 ring_wr; > + bool stopping; /* set at teardown to abort ring-full waits */ > +}; > + > +struct mbedtee_rpc_transport_ctx { > + int rpc_notify_virq; > + struct msi_msg rpc_msi_msg; > +}; > + > +struct mbedtee_supp { > + struct mutex mutex; /* serializes supp request queue and active requests */ > + struct tee_context *ctx; > + bool shutting_down; > + struct list_head reqs; > + struct list_head active_reqs; > + struct completion reqs_c; > +}; > + > +struct mbedtee_device { > + u32 version; > + bool yield; > + struct device *dev; > + struct tee_device *teedev; > + struct tee_shm_pool *pool; > + struct mbedtee_supp supp; > + struct xarray rpc_calls; > + struct mutex rpc_calls_mutex; /* serializes submission and shutdown */ > + bool stopping; > + atomic_long_t rpc_call_seq; > + const struct rpc_transport_ops *rpc_ops; > + struct mbedtee_t2r_ctx t2r; > + struct mbedtee_r2t_ctx r2t; > + struct mbedtee_rpc_transport_ctx transport; > +}; > + > +struct mbedtee_rpc_call { > + struct completion c; > + phys_addr_t rpc_phys; > + u32 state; > + /* One reference for the caller, one for the XArray entry. */ > + refcount_t refs; > + /* > + * Wire-format rpc_cmd with inline data[] payload. > + * MUST be the last field: rpc.data[] is a flexible array. > + */ > + struct rpc_cmd rpc; > +}; > + > +struct mbedtee_session { > + struct list_head list_node; > + u32 session_id; > + bool closing; > +}; > + > +struct mbedtee_context_data { > + bool is_supp_ctx; > + struct mutex mutex; /* serializes session list updates */ > + struct list_head sess_list; > +}; > + > +struct mbedtee_session * > +mbedtee_find_session_locked(struct mbedtee_context_data *ctxdata, > + u32 session_id); > + > +static inline phys_addr_t mbedtee_virt_to_phys(void *va) > +{ > + if (is_vmalloc_or_module_addr(va)) > + return page_to_phys(vmalloc_to_page(va)) + > + offset_in_page(va); > + > + return virt_to_phys(va); > +} > + > +int mbedtee_get_resource(struct device_node *node, > + const char *name, struct resource *res); > + > +/* > + * RPC transport operations -- implemented per-architecture. > + * > + * Caller (rpc_caller_{arm,riscv}.c) platform differences: > + * fastcall: ARM/ARM64 - direct SMC (arm_smccc_smc), synchronous, > + * never returns -ENOSPC. > + * RISC-V - writes rpc_cmd phys addr to the r2t ring > + * buffer, then waits for TEE completion via the t2r > + * ring. Despite the "fastcall" name this is a > + * synchronous ring-buffer round-trip; it may return > + * -ENOSPC when the r2t ring is full, and the caller > + * must retry (see rpc_routine() for the > + * MBEDTEE_RPC_COMPLETE_TEE retry loop). > + * yieldcall: ARM/ARM64 - one SMC posting rpc_cmd phys addr, waits > + * for completion. > + * RISC-V - writes rpc_cmd phys addr to r2t ring, waits > + * for completion. > + * Callee (rpc_callee_{arm,riscv}.c) platform differences: > + * notify: ARM/ARM64 - GIC SPI; the TEE broadcasts via GIC > + * hardware (GICv2 ITARGETS=0xFF, GICv3 IROUTER.IRM=1). > + * Ring metadata unchanged on CPU affinity events. > + * RISC-V - IMSIC MSI; the driver writes callee_hartid > + * and callee_imsic_id into the ring so the TEE knows > + * where to send the MSI. The affinity handler updates > + * both ring fields when the callee CPU goes offline. > + * > + * init: Set up the interrupt delivery mechanism (MSI on RISC-V, > + * GIC/OF-IRQ on ARM/ARM64) and, on RISC-V, write > + * ring->callee_hartid and ring->callee_imsic_id. @handler > + * is the hard-IRQ handler that drains the ring buffer and > + * dispatches RPC commands. > + * uninit: Tear down and free all interrupt resources. > + * update_affinity: Migrate T2R interrupt delivery to @new_cpu. Called by the > + * common cpuhp handler when the current callee CPU goes > + * offline. On RISC-V IMSIC, must update ring->callee_hartid > + * and ring->callee_imsic_id. ARM uses GIC hardware routing > + * and requires no ring update. > + * > + * The getter is implemented per-architecture in rpc_callee_{arm,riscv}.c; > + * only one is linked per build. > + */ > +struct rpc_transport_ops { > + int (*init)(struct mbedtee_device *mbedtee, struct rpc_ringbuf *ring, > + irq_handler_t handler); > + void (*uninit)(struct mbedtee_device *mbedtee); > + int (*update_affinity)(struct mbedtee_device *mbedtee, > + unsigned int new_cpu); > +}; > + > +const struct rpc_transport_ops *mbedtee_get_rpc_transport_ops(void); > + > +/* rpc-callee.c */ > +int mbedtee_rpc_init(struct mbedtee_device *mbedtee); > +void mbedtee_rpc_stop(struct mbedtee_device *mbedtee); > +void mbedtee_rpc_uninit(struct mbedtee_device *mbedtee); > +irqreturn_t mbedtee_rpc_irq_handler(int irq, void *dev_id); > + > +/* rpc_caller.c */ > +int mbedtee_rpc_call_alloc(struct mbedtee_device *mbedtee, > + size_t payload_size, struct mbedtee_rpc_call **call); > +void mbedtee_rpc_call_free(struct mbedtee_device *mbedtee, > + struct mbedtee_rpc_call *call); > +void mbedtee_rpc_complete_call(struct mbedtee_device *mbedtee, > + u64 waiter_id); > +void mbedtee_rpc_calls_abort(struct mbedtee_device *mbedtee); > +void mbedtee_rpc_calls_release(struct mbedtee_device *mbedtee); > +long mbedtee_rpc_wait_for_completion(struct mbedtee_device *mbedtee, > + struct mbedtee_rpc_call *call, bool killable); > +long mbedtee_rpc_wait_for_completion_timeout(struct mbedtee_device *mbedtee, > + struct mbedtee_rpc_call *call, > + unsigned long timeout); > +int mbedtee_open_session(struct tee_context *ctx, > + struct tee_ioctl_open_session_arg *arg, > + struct tee_param *param); > +int mbedtee_close_session(struct tee_context *ctx, u32 session); > +int mbedtee_invoke_func(struct tee_context *ctx, > + struct tee_ioctl_invoke_arg *arg, > + struct tee_param *param); > +int mbedtee_cancel_req(struct tee_context *ctx, u32 cancel_id, > + u32 session); > +int mbedtee_shm_register(struct tee_context *ctx, struct tee_shm *shm, > + struct page **pages, size_t num_pages, unsigned long start); > +int mbedtee_shm_unregister(struct tee_context *ctx, struct tee_shm *shm); > + > +/* rpc-caller-{arm,riscv}.c */ > +long mbedtee_rpc_yieldcall(struct mbedtee_device *mbedtee, > + unsigned long fn, struct mbedtee_rpc_call *call, > + bool interruptible); > +long mbedtee_rpc_fastcall(struct mbedtee_device *mbedtee, > + unsigned long fn, unsigned long a0, > + unsigned long a1, unsigned long a2); > +int mbedtee_r2t_init(struct mbedtee_device *mbedtee); > +void mbedtee_r2t_stop(struct mbedtee_device *mbedtee); > +void mbedtee_r2t_uninit(struct mbedtee_device *mbedtee); > + > +/* supp.c */ > +void mbedtee_supp_init(struct mbedtee_supp *supp); > +void mbedtee_supp_uninit(struct mbedtee_supp *supp); > +void mbedtee_supp_release(struct mbedtee_supp *supp, struct tee_context *ctx); > +void mbedtee_supp_abort_all(struct mbedtee_supp *supp); > +void mbedtee_supp_handler(struct mbedtee_device *mbedtee, > + u32 func, void *data, size_t size); > +int mbedtee_supp_recv(struct tee_context *ctx, u32 *func, > + u32 *num_params, struct tee_param *param); > +int mbedtee_supp_send(struct tee_context *ctx, u32 ret, > + u32 num_params, struct tee_param *param); > + > +#endif /* MBEDTEE_DRV_H */ > diff --git a/drivers/tee/mbedtee/mbedtee_msg.h b/drivers/tee/mbedtee/mbedtee_msg.h > new file mode 100644 > index 0000000..deb2d8c > --- /dev/null > +++ b/drivers/tee/mbedtee/mbedtee_msg.h > @@ -0,0 +1,219 @@ > +/* SPDX-License-Identifier: GPL-2.0-only */ > +/* > + * Copyright (c) 2020 Xing Loong <[email protected]> > + * > + * MbedTEE RPC wire protocol between REE (Linux) and TEE (MbedTEE OS). > + * > + * This header is the Linux-kernel mirror of > + * mbedtee-os/drivers/rpc/include/rpc/rpc.h. Both copies MUST define > + * identical struct layouts. The static_assert() checks at the bottom > + * of this file (and the _Static_assert() checks in rpc.h) enforce this > + * contract at compile time on every architecture. > + * > + * Type mapping (wire -> Linux kernel alias): > + * uint8_t -> u8 -- boolean flag / padding byte > + * uint16_t -> u16 -- byte-count field (< 64 KiB payload limit) > + * uint32_t -> u32 -- 32-bit ID, counter, flag, or enum > + * int32_t -> s32 -- function-defined signed result code > + * uint64_t -> u64 -- physical address, kernel pointer token, or size > + * > + * Using u64 for all pointer-sized fields eliminates ABI width mismatches > + * in mixed-width configurations (e.g. RV64 REE + RV32 TEE, or ARM32-LPAE > + * REE + ARM32 TEE). It also prevents phys_addr_t/unsigned-long confusion > + * when CONFIG_PHYS_ADDR_T_64BIT=y on a 32-bit REE kernel. > + */ > + > +#ifndef MBEDTEE_MSG_H > +#define MBEDTEE_MSG_H > + > +#include <linux/bits.h> > +#include <linux/build_bug.h> > +#include <linux/types.h> > + > +/* > + * Fastcall / yield-call classification > + */ > +#define MBEDTEE_RPC_FASTCALL BIT(31) > +#define MBEDTEE_RPC_IS_FASTCALL(fn) ((fn) & MBEDTEE_RPC_FASTCALL) > + > +/* > + * REE -> TEE fast calls > + */ > +#define MBEDTEE_RPC_OS_VERSION (0x0100 | MBEDTEE_RPC_FASTCALL) > +#define MBEDTEE_RPC_SUPPORT_YIELD (0x0101 | MBEDTEE_RPC_FASTCALL) > +#define MBEDTEE_RPC_COMPLETE_TEE (0x0102 | MBEDTEE_RPC_FASTCALL) > + > +/* > + * REE -> TEE yield calls > + */ > +#define MBEDTEE_RPC_OPEN_SESSION 1 > +#define MBEDTEE_RPC_INVOKE_SESSION 2 > +#define MBEDTEE_RPC_CLOSE_SESSION 3 > +#define MBEDTEE_RPC_REGISTER_SHM 5 > +#define MBEDTEE_RPC_UNREGISTER_SHM 6 > +#define MBEDTEE_RPC_CANCEL 7 > + > +/* > + * TEE -> REE RPC calls > + */ > +#define MBEDTEE_RPC_COMPLETE_REE 0 > +#define MBEDTEE_RPC_REETIME 1 > +#define MBEDTEE_RPC_REEFS 2 > +#define MBEDTEE_RPC_RPMB 3 > +#define MBEDTEE_RPC_MAX 4 > + > +/* > + * Supplicant types (TEE -> REE -> tee-supplicant) > + */ > +#define MBEDTEE_SUPP_REEFS 1 > +#define MBEDTEE_SUPP_RPMB 2 > + > +/* MbedTEE RPC protocol uses 4 KiB page units regardless of host PAGE_SIZE. */ > +#define MBEDTEE_PAGE_SIZE 4096UL > + > +/* > + * REE <-> TEE RPC call command (wire format, 32 bytes fixed header). > + * > + * Fixed layout: > + * +0 id u32 - RPC function identifier > + * +4 size u16 - inline payload byte count (0..65535) > + * +6 interrupted u8 - set by REE when caller is interrupted > + * +7 reserved u8 - must be zero (explicit alignment pad) > + * +8 ret s32 - return value written by callee > + * +12 pad u32 - must be zero (explicit alignment pad) > + * +16 waiter_id u64 - sync RPC request ID echoed on completion > + * +24 shm u64 - physical address of sync-RPC shared memory > + * +32 data[] u64 - inline payload (waiter_id==0) or empty > + * > + * Yield-call contract: > + * - Session/control RPCs return GlobalPlatform result codes here. > + * - Host-local errno values must be translated before being put on the wire. > + * - Fast calls may use function-specific return values. > + */ > +struct rpc_cmd { > + u32 id; > + u16 size; > + u8 interrupted; > + u8 reserved; > + s32 ret; > + u32 pad; > + u64 waiter_id; > + u64 shm; > + u64 data[]; > +}; > + > +/* > + * REE -> TEE pages for GP shared memory. > + * > + * All fields are u64 so the layout is identical when REE is 64-bit > + * but TEE is 32-bit (physical addresses, sizes and counts use 64 bits). > + */ > +struct rpc_memref { > + u64 id; > + u64 pages; > + u64 offset; > + u64 size; > + u64 cnt; > +}; > + > +/* > + * REE -> TEE parameter entry: value or shared memory reference. > + */ > +union rpc_tee_param { > + struct rpc_memref memref; > + > + struct { > + u32 a; > + u32 b; > + } value; > +}; > + > +/* > + * REE -> TEE parameters for RPC session operations > + * (MBEDTEE_RPC_OPEN_SESSION / MBEDTEE_RPC_INVOKE_SESSION / > + * MBEDTEE_RPC_CLOSE_SESSION). > + */ > +struct rpc_param { > + s32 session_id; > + u32 cmd_id; > + u32 ret_origin; > + u32 params_type; > + union rpc_tee_param params[4]; > + u8 uuid[16]; > + u8 clnt_uuid[16]; > +}; > + > +/* > + * REE -> TEE cancellation request (MBEDTEE_RPC_CANCEL). > + */ > +struct rpc_cancel_req { > + u32 session_id; > + u32 cancel_id; > +}; > + > +/* > + * TEE <-> REE supplicant payloads for REEFS and RPMB RPCs. > + * Only the fixed headers are interpreted in the Linux driver. > + */ > +/* > + * Common supplicant command header. Both reefs_cmd and rpmb_cmd embed this > + * as their first member so the kernel can write a status code through a > + * single cast to (struct supp_cmd_hdr *) without caring about the full > + * payload type. The wire layout is identical on every architecture. > + */ > +struct supp_cmd_hdr { > + s32 ret; > + s32 op; > +}; > + > +struct reefs_cmd { > + struct supp_cmd_hdr hdr; > + s32 flags; > + s32 fd; > + u64 len; > + u8 data[]; > +}; > + > +struct rpmb_cmd { > + struct supp_cmd_hdr hdr; > + u32 nframes; > + u8 data[]; > +}; > + > +/* > + * TEE <-> REE RPC ring buffer header (24 bytes fixed). > + */ > +struct rpc_ringbuf { > + u32 wr; /* producer write pointer */ > + u32 rd; /* consumer read pointer */ > + u32 callee_ready; /* callee ready flag */ > + u32 callee_imsic_id; /* RISC-V only: IMSIC local interrupt id */ > + u32 callee_hartid; /* RISC-V only: target hart-id for T2R notification */ > + u32 reserved; /* padding, must be zero */ > + u8 mem[]; > +}; > + > +/* > + * Compile-time ABI layout assertions. > + * These must match the _Static_assert() checks in mbedtee-os rpc/rpc.h. > + */ > +static_assert(sizeof(struct rpc_cmd) == 32, > + "rpc_cmd wire size mismatch"); > +static_assert(offsetof(struct rpc_cmd, ret) == 8, > + "rpc_cmd.ret offset mismatch"); > +static_assert(offsetof(struct rpc_cmd, waiter_id) == 16, > + "rpc_cmd.waiter_id offset mismatch"); > +static_assert(offsetof(struct rpc_cmd, shm) == 24, > + "rpc_cmd.shm offset mismatch"); > +static_assert(offsetof(struct rpc_cmd, data) == 32, > + "rpc_cmd.data offset mismatch"); > +static_assert(sizeof(struct rpc_memref) == 40, > + "rpc_memref wire size mismatch"); > +static_assert(sizeof(struct rpc_ringbuf) == 24, > + "rpc_ringbuf header size mismatch"); > +static_assert(sizeof(struct rpc_param) == 208, > + "rpc_param wire size mismatch"); > +static_assert(sizeof(struct rpc_cancel_req) == 8, > + "rpc_cancel_req wire size mismatch"); > + > +#endif /* MBEDTEE_MSG_H */ > diff --git a/drivers/tee/mbedtee/rpc_callee.c b/drivers/tee/mbedtee/rpc_callee.c > new file mode 100644 > index 0000000..258f8d1 > --- /dev/null > +++ b/drivers/tee/mbedtee/rpc_callee.c > @@ -0,0 +1,697 @@ > +// SPDX-License-Identifier: GPL-2.0-only > +/* > + * Copyright (c) 2020 Xing Loong <[email protected]> > + * TEE->REE callee-side: handles interrupts from TEE, processes RPC requests. > + */ > +#include <linux/cpu.h> > +#include <linux/cpuhotplug.h> > +#include <linux/cpumask.h> > +#include <linux/delay.h> > +#include <linux/errno.h> > +#include <linux/interrupt.h> > +#include <linux/io.h> > +#include <linux/jiffies.h> > +#include <linux/ktime.h> > +#include <linux/module.h> > +#include <linux/of.h> > +#include <linux/slab.h> > +#include <linux/spinlock.h> > +#include <linux/tee_drv.h> > +#include <linux/workqueue.h> > + > +#include "mbedtee_drv.h" > + > +/* Give up completing a sync RPC if the TEE stops draining its ring. */ > +#define MBEDTEE_RPC_COMPLETE_TIMEOUT (30 * HZ) > + > +/* > + * CPU hotplug state shared across all mbedtee instances (registered once). > + */ > +static int mbedtee_cpuhp_state = -EINVAL; > +static unsigned int mbedtee_cpuhp_instances; > +static DEFINE_MUTEX(mbedtee_cpuhp_mutex); > +/* rpc_routine() runs on workqueue, re-drains ring on completion */ > +static void rpc_routine(struct work_struct *work); > + > +/* > + * Common CPU-online handler: add the new CPU into the eligible T2R > + * delivery set and let the transport re-apply affinity. > + */ > +static int mbedtee_cpu_online(unsigned int cpu, struct hlist_node *node) > +{ > + struct mbedtee_t2r_ctx *ctx = > + hlist_entry(node, struct mbedtee_t2r_ctx, hp_node); > + struct mbedtee_device *mbedtee = > + container_of(ctx, struct mbedtee_device, t2r); > + unsigned int target_cpu; > + > + if (!ctx->t2r_ring || cpumask_test_cpu(cpu, &ctx->callee_cpus)) > + return 0; > + > + cpumask_set_cpu(cpu, &ctx->callee_cpus); > + target_cpu = cpumask_first(&ctx->callee_cpus); > + > + if (mbedtee->rpc_ops && mbedtee->rpc_ops->update_affinity) > + mbedtee->rpc_ops->update_affinity(mbedtee, target_cpu); > + > + return 0; > +} > + > +/* > + * Common CPU-offline handler: migrates T2R interrupt delivery to another > + * online CPU when the current callee CPU is being taken offline. > + * > + * The transport-specific update_affinity() callback handles the actual > + * interrupt migration (e.g. irq_set_affinity + callee_hartid update for > + * RISC-V IMSIC; irq_set_affinity only for ARM - no ring update needed). > + */ > +static int mbedtee_cpu_offline(unsigned int cpu, struct hlist_node *node) > +{ > + struct mbedtee_t2r_ctx *ctx = > + hlist_entry(node, struct mbedtee_t2r_ctx, hp_node); > + struct mbedtee_device *mbedtee = > + container_of(ctx, struct mbedtee_device, t2r); > + unsigned int new_cpu; > + > + if (!ctx->t2r_ring || !cpumask_test_cpu(cpu, &ctx->callee_cpus)) > + return 0; > + > + /* > + * Remove the dying CPU from the eligible set. If the set is now > + * empty (all CPUs going away at once -- highly unlikely but safe), > + * fall back to CPU 0 which Linux guarantees to be the last offline. > + */ > + cpumask_clear_cpu(cpu, &ctx->callee_cpus); > + if (cpumask_empty(&ctx->callee_cpus)) > + cpumask_set_cpu(0, &ctx->callee_cpus); > + > + new_cpu = cpumask_first(&ctx->callee_cpus); > + > + if (mbedtee->rpc_ops && mbedtee->rpc_ops->update_affinity) > + mbedtee->rpc_ops->update_affinity(mbedtee, new_cpu); > + return 0; > +} > + > +static inline size_t t2r_available_size(struct mbedtee_t2r_ctx *ctx) > +{ > + u32 wr; > + u32 rd; > + u32 shm_size = ctx->t2r_ring_sz; > + > + /* Pair with producer release store after writing ring payload. */ > + wr = smp_load_acquire(&ctx->t2r_ring->wr); > + rd = READ_ONCE(ctx->t2r_ring_rd); > + > + /* Corrupt indices mean the stream cannot be trusted. */ > + if (wr > shm_size || rd > shm_size) > + return 0; > + > + if (wr >= rd) > + return wr - rd; > + return shm_size + wr - rd; > +} > + > +static void t2r_ring_copy(struct mbedtee_t2r_ctx *ctx, > + void *data, size_t size, u32 rd) > +{ > + u32 remain; > + u32 shm_size = ctx->t2r_ring_sz; > + struct rpc_ringbuf *shm = ctx->t2r_ring; > + > + if (rd + size <= shm_size) { > + memcpy(data, &shm->mem[rd], size); > + } else { > + remain = rd + size - shm_size; > + memcpy(data, &shm->mem[rd], size - remain); > + memcpy((u8 *)data + size - remain, > + &shm->mem[0], remain); > + } > +} > + > +static void t2r_ring_advance(struct mbedtee_t2r_ctx *ctx, size_t size) > +{ > + u32 rd = READ_ONCE(ctx->t2r_ring_rd); > + u32 shm_size = ctx->t2r_ring_sz; > + struct rpc_ringbuf *shm = ctx->t2r_ring; > + > + if (rd + size > shm_size) > + rd = rd + size - shm_size; > + else > + rd += size; > + > + WRITE_ONCE(ctx->t2r_ring_rd, rd); > + /* Publish updated consumer index after command parsing is complete. */ > + smp_store_release(&shm->rd, rd); > +} > + > +/* > + * Read and consume @size bytes from the TEE-to-REE ring buffer. > + * Must be called with ctx->ring_lock held. > + */ > +static void t2r_ring_read(struct mbedtee_t2r_ctx *ctx, > + void *data, size_t size) > +{ > + t2r_ring_copy(ctx, data, size, READ_ONCE(ctx->t2r_ring_rd)); > + t2r_ring_advance(ctx, size); > +} > + > +static bool rpc_queue_complete_only(struct mbedtee_t2r_ctx *ctx, > + u64 waiter_id) > +{ > + /* Called with ctx->ring_lock held. */ > + if (ctx->complete_work_pending) > + return false; > + > + ctx->complete_work_pending = true; > + ctx->complete_work.waiter_id = waiter_id; > + queue_work(ctx->rpc_wq, &ctx->complete_work.work); > + > + return true; > +} > + > +/* > + * Try to pick the next RPC command from the ring buffer. > + * > + * For asynchronous RPCs the data payload follows the cmd header in the > + * ring. If the payload has not fully arrived yet the cmd header is still > + * consumed (it was already read) but the entry is marked incomplete so > + * that the next IRQ resumes reading the payload instead of re-reading > + * a header. > + * > + * For synchronous RPCs (waiter_id != 0) the work descriptor is allocated > + * BEFORE the ring header is consumed. This guarantees that if the > + * allocator returns NULL the ring entry is left intact so the IRQ > + * handler will retry on the next wake-up rather than leaving a TEE > + * thread blocked with no response. > + * > + * Returns a work descriptor on success, or NULL when no complete entry > + * is available. > + */ > +static struct rpc_work *rpc_pick_next(struct mbedtee_device *mbedtee, > + struct mbedtee_t2r_ctx *ctx, struct rpc_work *c) > +{ > + struct rpc_cmd cmd; > + void (*func)(struct mbedtee_device *mbedtee, void *data, size_t size); > + struct rpc_work *new_work = NULL; > + resource_size_t off; > + phys_addr_t shm_phys; > + u64 shm_wire; > + > + /* Resume reading payload of a previously incomplete async RPC */ > + if (ctx->pending_async) { > + if (t2r_available_size(ctx) < ctx->pending_size) > + return NULL; > + ctx->pending_async = false; > + t2r_ring_read(ctx, c->data, ctx->pending_size); > + c->func = ctx->pending_func; > + c->size = ctx->pending_size; > + c->waiter_id = 0; > + return c; > + } > + > + if (t2r_available_size(ctx) < sizeof(cmd)) > + return NULL; > + > + /* > + * Peek at the header without advancing the ring pointer. > + * For sync RPCs we must pre-allocate before consuming the entry. > + */ > + memset(&cmd, 0, sizeof(cmd)); > + t2r_ring_copy(ctx, &cmd, sizeof(cmd), READ_ONCE(ctx->t2r_ring_rd)); > + > + if (cmd.id >= MBEDTEE_RPC_MAX) > + goto skip; > + > + func = ctx->rpc_handlers[cmd.id]; > + if (!func) > + goto skip; > + > + /* > + * The TEE sends COMPLETE_REE asynchronously and the supplicant > + * commands (REEFS/RPMB) synchronously. Reject inverted usage so > + * that blocking supplicant handlers can never run inline in IRQ > + * context while the ring lock is held. > + */ > + switch (cmd.id) { > + case MBEDTEE_RPC_COMPLETE_REE: > + if (cmd.waiter_id != 0) > + goto skip; > + break; > + case MBEDTEE_RPC_REETIME: > + case MBEDTEE_RPC_REEFS: > + case MBEDTEE_RPC_RPMB: > + if (cmd.waiter_id == 0) > + goto skip; > + if ((cmd.id == MBEDTEE_RPC_REEFS || > + cmd.id == MBEDTEE_RPC_RPMB) && > + cmd.size < sizeof(struct supp_cmd_hdr)) > + goto skip; > + break; > + default: > + break; > + } > + > + if (cmd.waiter_id == 0) { > + /* Async RPC: payload follows header in the ring */ > + if (cmd.size > sizeof(ctx->rpc_data)) > + goto skip; > + > + /* Consume the header now that we know it is valid */ > + t2r_ring_advance(ctx, sizeof(cmd)); > + > + if (t2r_available_size(ctx) < cmd.size) { > + /* Mark as incomplete, resume on next iteration */ > + ctx->pending_async = true; > + ctx->pending_size = cmd.size; > + ctx->pending_func = func; > + return NULL; > + } > + > + t2r_ring_read(ctx, c->data, cmd.size); > + } else { > + /* Sync RPC: data lives in the shared memory region */ > + shm_wire = cmd.shm; > + shm_phys = (phys_addr_t)shm_wire; > + if ((u64)shm_phys != shm_wire) > + goto skip; > + > + if (shm_phys < ctx->t2r_shm_phys) > + goto skip; > + > + off = shm_phys - ctx->t2r_shm_phys; > + if (cmd.size == 0 || cmd.size > ctx->t2r_shm_sz) > + goto skip; > + > + if (off > ctx->t2r_shm_sz - cmd.size) > + goto skip; > + > + new_work = kzalloc_obj(*new_work, GFP_ATOMIC); > + if (!new_work) { > + /* > + * Leave the header in place. The TEE re-notifies when > + * its sync RPC retry timer fires, so this entry is > + * retried on the next IRQ instead of being lost. > + */ > + return NULL; > + } > + > + t2r_ring_advance(ctx, sizeof(cmd)); > + > + c = new_work; > + c->mbedtee = mbedtee; > + c->data = ctx->t2r_shm + off; > + } > + > + c->func = func; > + c->waiter_id = cmd.waiter_id; > + c->size = cmd.size; > + > + return c; > + > +skip: > + /* > + * If this was a sync RPC (waiter_id != 0) we must notify the TEE > + * thread so it does not wait forever. Defer the completion fastcall > + * to process context because transport implementations may sleep. > + * If the single deferred-completion slot is already in use, leave the > + * header in place and retry after the pending completion has drained. > + */ > + if (cmd.waiter_id != 0 && !rpc_queue_complete_only(ctx, cmd.waiter_id)) > + return NULL; > + > + /* > + * Bad or unhandled entry: consume header plus any inline payload > + * (async RPC payloads follow the header in the ring) to keep the > + * ring moving. For sync RPCs payload is in t2r_shm, not the ring. > + * > + * An async payload must fit the ring and be fully published; > + * otherwise the stream cannot be framed and no later entry can be > + * trusted, so drop the whole pending snapshot instead of moving > + * the reader past the producer. > + */ > + if (cmd.waiter_id == 0 && > + (cmd.size > ctx->t2r_ring_sz - sizeof(cmd) || > + t2r_available_size(ctx) < sizeof(cmd) + cmd.size)) { > + t2r_ring_advance(ctx, t2r_available_size(ctx)); > + return NULL; > + } > + > + t2r_ring_advance(ctx, sizeof(cmd) + (cmd.waiter_id ? 0 : cmd.size)); > + > + return NULL; > +} > + > +static void rpc_drain_ring(struct mbedtee_device *mbedtee) > +{ > + struct mbedtee_t2r_ctx *ctx = &mbedtee->t2r; > + struct rpc_work rw = {}, *c = NULL; > + unsigned long flags; > + > + if (!ctx->t2r_ring) > + return; > + > + spin_lock_irqsave(&ctx->ring_lock, flags); > + while (READ_ONCE(ctx->t2r_ring_rd) != > + /* Pair with producer store-release after ring write. */ > + smp_load_acquire(&ctx->t2r_ring->wr)) { > + rw.data = ctx->rpc_data; > + c = rpc_pick_next(mbedtee, ctx, &rw); > + if (!c) > + break; > + > + if (c->waiter_id == 0) { > + c->func(mbedtee, c->data, c->size); > + } else { > + INIT_WORK(&c->work, rpc_routine); > + queue_work(ctx->rpc_wq, &c->work); > + } > + } > + spin_unlock_irqrestore(&ctx->ring_lock, flags); > +} > + > +static void rpc_routine(struct work_struct *work) > +{ > + struct rpc_work *c = container_of(work, struct rpc_work, work); > + struct mbedtee_t2r_ctx *ctx = &c->mbedtee->t2r; > + unsigned long deadline = jiffies + MBEDTEE_RPC_COMPLETE_TIMEOUT; > + unsigned long flags; > + long ret; > + > + if (!c->complete_only) > + c->func(c->mbedtee, c->data, c->size); > + > + /* > + * COMPLETE_TEE must reach the TEE thread that issued the sync RPC. > + * On RISC-V the notification uses a shared ring buffer that may be > + * transiently full when many RPCs are in flight concurrently. > + * Keep retrying rather than silently dropping the completion, which > + * would leave the TEE thread blocked in rpc_call_sync forever. > + * On ARM the fastcall is a direct SMC so -ENOSPC never occurs. > + * Stop retrying at teardown or when the deadline passes so a dead > + * TEE cannot hang destroy_workqueue(). > + */ > + do { > + if (READ_ONCE(ctx->stopping)) > + break; > + ret = mbedtee_rpc_fastcall(c->mbedtee, MBEDTEE_RPC_COMPLETE_TEE, > + (unsigned long)c->waiter_id, 0, 0); > + if (ret != -ENOSPC && ret != -ENOMEM) > + break; > + if (time_after_eq(jiffies, deadline)) { > + dev_err(c->mbedtee->dev, > + "COMPLETE_TEE for waiter %llu timed out\n", > + c->waiter_id); > + break; > + } > + cond_resched(); > + } while (1); > + > + if (c != &ctx->complete_work) { > + kfree(c); > + return; > + } > + > + /* complete_work_pending is always accessed under ring_lock. */ > + spin_lock_irqsave(&ctx->ring_lock, flags); > + ctx->complete_work_pending = false; > + spin_unlock_irqrestore(&ctx->ring_lock, flags); > + if (!READ_ONCE(ctx->stopping)) > + rpc_drain_ring(c->mbedtee); > +} > + > +/* > + * Hard IRQ handler -- drains the TEE-to-REE ring buffer. > + * > + * Asynchronous RPCs (waiter_id == 0) are handled inline since all async > + * handlers (complete(), ktime_get_real_ts64()) are non-blocking. > + * > + * Synchronous RPCs (waiter_id != 0) may block waiting for the > + * tee-supplicant, so they are dispatched to a workqueue. > + */ > +irqreturn_t mbedtee_rpc_irq_handler(int irq, void *dev_id) > +{ > + struct mbedtee_device *mbedtee = dev_id; > + > + if (!mbedtee) > + return IRQ_NONE; > + > + rpc_drain_ring(mbedtee); > + > + return IRQ_HANDLED; > +} > + > +static void mbedtee_rpc_complete(struct mbedtee_device *mbedtee, > + void *data, size_t size) > +{ > + u64 waiter_id; > + > + if (size < sizeof(waiter_id)) > + return; > + > + memcpy(&waiter_id, data, sizeof(waiter_id)); > + if (!waiter_id) > + return; > + > + mbedtee_rpc_complete_call(mbedtee, waiter_id); > +} > + > +static void mbedtee_ree_time(struct mbedtee_device *mbedtee, > + void *data, size_t size) > +{ > + struct timespec64 *ts = data; > + > + if (size < sizeof(*ts)) > + return; > + > + ktime_get_real_ts64(ts); > +} > + > +static void mbedtee_reefs_supp(struct mbedtee_device *mbedtee, > + void *data, size_t size) > +{ > + mbedtee_supp_handler(mbedtee, MBEDTEE_SUPP_REEFS, data, size); > +} > + > +static void mbedtee_rpmb_supp(struct mbedtee_device *mbedtee, > + void *data, size_t size) > +{ > + mbedtee_supp_handler(mbedtee, MBEDTEE_SUPP_RPMB, data, size); > +} > + > +static void mbedtee_register_rpc(struct mbedtee_device *mbedtee, > + u32 id, void (*func)(struct mbedtee_device *mbedtee, > + void *data, size_t size)) > +{ > + struct mbedtee_t2r_ctx *ctx = &mbedtee->t2r; > + > + if (WARN_ON(id >= MBEDTEE_RPC_MAX || !func)) > + return; > + > + ctx->rpc_handlers[id] = func; > +} > + > +int mbedtee_rpc_init(struct mbedtee_device *mbedtee) > +{ > + struct mbedtee_t2r_ctx *ctx = &mbedtee->t2r; > + struct device_node *node = mbedtee->dev->of_node; > + struct resource res; > + bool cpuhp_state_created = false; > + int ret; > + > + memset(ctx, 0, sizeof(*ctx)); > + spin_lock_init(&ctx->ring_lock); > + mbedtee->rpc_ops = mbedtee_get_rpc_transport_ops(); > + if (!mbedtee->rpc_ops) > + return -EIO; > + > + ret = mbedtee_get_resource(node, "t2r-ring", &res); > + if (ret) { > + dev_err(mbedtee->dev, "t2r-ring not found in DT\n"); > + return ret; > + } > + > + ctx->t2r_ring = memremap(res.start, resource_size(&res), > + MEMREMAP_WB); > + if (!ctx->t2r_ring) { > + dev_err(mbedtee->dev, "failed to map t2r ring\n"); > + return -ENOMEM; > + } > + if (resource_size(&res) <= sizeof(struct rpc_ringbuf)) { > + dev_err(mbedtee->dev, "t2r-ring too small\n"); > + memunmap(ctx->t2r_ring); > + ctx->t2r_ring = NULL; > + return -EINVAL; > + } > + ctx->t2r_ring_sz = resource_size(&res) - sizeof(struct rpc_ringbuf); > + /* Observe latest producer index only after ring metadata is visible. */ > + WRITE_ONCE(ctx->t2r_ring_rd, smp_load_acquire(&ctx->t2r_ring->rd)); > + > + dev_dbg(mbedtee->dev, "t2r-ring %pa\n", &res.start); > + > + ret = mbedtee_get_resource(node, "t2r-shm", &res); > + if (ret) { > + dev_err(mbedtee->dev, "t2r-shm not found in DT\n"); > + goto err_ring; > + } > + > + ctx->t2r_shm_phys = res.start; > + ctx->t2r_shm_sz = resource_size(&res); > + ctx->t2r_shm = memremap(res.start, ctx->t2r_shm_sz, > + MEMREMAP_WB); > + if (!ctx->t2r_shm) { > + dev_err(mbedtee->dev, "failed to map t2r shm\n"); > + ret = -ENOMEM; > + goto err_ring; > + } > + > + dev_dbg(mbedtee->dev, "t2r-shm %pa\n", &res.start); > + > + ctx->rpc_wq = alloc_workqueue("mbedtee-rpc", WQ_UNBOUND, 0); > + if (!ctx->rpc_wq) { > + dev_err(mbedtee->dev, "failed to create rpc workqueue\n"); > + ret = -ENOMEM; > + goto err_shm; > + } > + > + ctx->complete_work.mbedtee = mbedtee; > + ctx->complete_work.complete_only = true; > + INIT_WORK(&ctx->complete_work.work, rpc_routine); > + > + mbedtee_register_rpc(mbedtee, MBEDTEE_RPC_COMPLETE_REE, mbedtee_rpc_complete); > + mbedtee_register_rpc(mbedtee, MBEDTEE_RPC_REETIME, mbedtee_ree_time); > + mbedtee_register_rpc(mbedtee, MBEDTEE_RPC_REEFS, mbedtee_reefs_supp); > + mbedtee_register_rpc(mbedtee, MBEDTEE_RPC_RPMB, mbedtee_rpmb_supp); > + > + /* > + * Register cpuhp instance BEFORE enabling interrupts so that > + * T2R delivery migration is in place before the first IRQ fires. > + */ > + mutex_lock(&mbedtee_cpuhp_mutex); > + if (mbedtee_cpuhp_state < 0) { > + mbedtee_cpuhp_state = > + cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, > + "tee/mbedtee:rpc-callee", > + mbedtee_cpu_online, > + mbedtee_cpu_offline); > + if (mbedtee_cpuhp_state < 0) { > + ret = mbedtee_cpuhp_state; > + mutex_unlock(&mbedtee_cpuhp_mutex); > + goto err_wq; > + } > + cpuhp_state_created = true; > + } > + > + ret = cpuhp_state_add_instance(mbedtee_cpuhp_state, &ctx->hp_node); > + if (!ret) { > + mbedtee_cpuhp_instances++; > + ctx->cpuhp_added = true; > + } else if (cpuhp_state_created) { > + cpuhp_remove_multi_state(mbedtee_cpuhp_state); > + mbedtee_cpuhp_state = -EINVAL; > + } > + mutex_unlock(&mbedtee_cpuhp_mutex); > + if (ret != 0) > + goto err_wq; > + > + ret = mbedtee->rpc_ops->init(mbedtee, ctx->t2r_ring, > + mbedtee_rpc_irq_handler); > + if (ret != 0) > + goto err_cpuhp; > + > + /* Signal the TEE that the REE callee is ready to receive. */ > + smp_store_release(&ctx->t2r_ring->callee_ready, true); > + > + return 0; > + > +err_cpuhp: > + mutex_lock(&mbedtee_cpuhp_mutex); > + if (mbedtee_cpuhp_state >= 0 && ctx->cpuhp_added) { > + cpuhp_state_remove_instance_nocalls(mbedtee_cpuhp_state, > + &ctx->hp_node); > + ctx->cpuhp_added = false; > + if (mbedtee_cpuhp_instances) > + mbedtee_cpuhp_instances--; > + if (!mbedtee_cpuhp_instances) { > + cpuhp_remove_multi_state(mbedtee_cpuhp_state); > + mbedtee_cpuhp_state = -EINVAL; > + } > + } > + mutex_unlock(&mbedtee_cpuhp_mutex); > +err_wq: > + destroy_workqueue(ctx->rpc_wq); > + ctx->rpc_wq = NULL; > +err_shm: > + memunmap(ctx->t2r_shm); > + ctx->t2r_shm = NULL; > +err_ring: > + memunmap(ctx->t2r_ring); > + ctx->t2r_ring = NULL; > + return ret; > +} > + > +void mbedtee_rpc_uninit(struct mbedtee_device *mbedtee) > +{ > + struct mbedtee_t2r_ctx *ctx; > + > + if (!mbedtee) > + return; > + > + ctx = &mbedtee->t2r; > + mbedtee_rpc_stop(mbedtee); > + > + /* Transport and workqueue resources can now be torn down safely. */ > + > + mutex_lock(&mbedtee_cpuhp_mutex); > + if (mbedtee_cpuhp_state >= 0 && ctx->cpuhp_added) { > + cpuhp_state_remove_instance_nocalls(mbedtee_cpuhp_state, > + &ctx->hp_node); > + ctx->cpuhp_added = false; > + if (mbedtee_cpuhp_instances) > + mbedtee_cpuhp_instances--; > + if (!mbedtee_cpuhp_instances) { > + cpuhp_remove_multi_state(mbedtee_cpuhp_state); > + mbedtee_cpuhp_state = -EINVAL; > + } > + } > + mutex_unlock(&mbedtee_cpuhp_mutex); > + > + if (mbedtee->rpc_ops) > + mbedtee->rpc_ops->uninit(mbedtee); > + > + if (ctx->rpc_wq) { > + destroy_workqueue(ctx->rpc_wq); > + ctx->rpc_wq = NULL; > + } > + > + if (ctx->t2r_shm) { > + memunmap(ctx->t2r_shm); > + ctx->t2r_shm = NULL; > + } > + > + if (ctx->t2r_ring) { > + memunmap(ctx->t2r_ring); > + ctx->t2r_ring = NULL; > + } > +} > + > +/* > + * Quiesce all RPC activity: reject new submissions and wake every waiter. > + * Idempotent -- called directly from remove() and again from rpc_uninit() > + * (which the probe error path reaches without a prior stop). > + */ > +void mbedtee_rpc_stop(struct mbedtee_device *mbedtee) > +{ > + struct mbedtee_t2r_ctx *ctx; > + > + if (!mbedtee) > + return; > + > + ctx = &mbedtee->t2r; > + /* Stop advertising the REE callee before interrupt teardown. */ > + if (ctx->t2r_ring) > + WRITE_ONCE(ctx->t2r_ring->callee_ready, false); > + WRITE_ONCE(ctx->stopping, true); > + mbedtee_r2t_stop(mbedtee); > + mbedtee_supp_abort_all(&mbedtee->supp); > + mbedtee_rpc_calls_abort(mbedtee); > +} > diff --git a/drivers/tee/mbedtee/rpc_callee_arm.c b/drivers/tee/mbedtee/rpc_callee_arm.c > new file mode 100644 > index 0000000..871bba5 > --- /dev/null > +++ b/drivers/tee/mbedtee/rpc_callee_arm.c > @@ -0,0 +1,91 @@ > +// SPDX-License-Identifier: GPL-2.0-only > +/* > + * Copyright (c) 2020 Xing Loong <[email protected]> > + * ARM/ARM64 GIC transport for the TEE->REE RPC callee. > + * > + * On ARM/ARM64 the TEE raises a rising-edge software-generated interrupt > + * described by the "interrupts" property of the mbedtee DT node. This file > + * obtains that virq via of_irq_get() and registers the hard IRQ handler > + * supplied by the core callee layer. > + */ > +#include <linux/interrupt.h> > +#include <linux/cpu.h> > +#include <linux/of.h> > +#include <linux/of_irq.h> > + > +#include "mbedtee_drv.h" > + > +static int arm_transport_init(struct mbedtee_device *mbedtee, > + struct rpc_ringbuf *ring, irq_handler_t handler) > +{ > + struct mbedtee_t2r_ctx *ctx = &mbedtee->t2r; > + struct mbedtee_rpc_transport_ctx *tctx = &mbedtee->transport; > + int ret; > + > + tctx->rpc_notify_virq = of_irq_get(mbedtee->dev->of_node, 0); > + if (tctx->rpc_notify_virq <= 0) > + return tctx->rpc_notify_virq ? tctx->rpc_notify_virq : -ENODEV; > + > + ret = request_irq(tctx->rpc_notify_virq, handler, > + 0, "mbedtee-rpc", mbedtee); > + if (ret) { > + dev_warn(mbedtee->dev, "request_irq %d failed: %d\n", > + tctx->rpc_notify_virq, ret); > + tctx->rpc_notify_virq = 0; > + return ret; > + } > + > + /* > + * Spread the T2R SPI across all online CPUs in Linux. > + * The TEE needs no CPU information from Linux: it broadcasts > + * via GIC hardware (GICv2 ITARGETS=0xFF / GICv3 IROUTER.IRM=1). > + */ > + cpumask_copy(&ctx->callee_cpus, cpu_online_mask); > + if (irq_set_affinity(tctx->rpc_notify_virq, &ctx->callee_cpus)) > + dev_warn(mbedtee->dev, "irq_set_affinity failed\n"); > + > + dev_dbg(mbedtee->dev, "t2r-irq %d\n", tctx->rpc_notify_virq); > + return 0; > +} > + > +static void arm_transport_uninit(struct mbedtee_device *mbedtee) > +{ > + struct mbedtee_rpc_transport_ctx *tctx = &mbedtee->transport; > + > + if (tctx->rpc_notify_virq > 0) { > + free_irq(tctx->rpc_notify_virq, mbedtee); > + tctx->rpc_notify_virq = 0; > + } > +} > + > +/* > + * ARM T2R uses a GIC SPI: the TEE broadcasts it via GIC hardware > + * (GICv2 ITARGETS=0xFF or GICv3 IROUTER.IRM=1) to any alive CPU. > + * TrustZone shares physical CPUs -- no ring update is ever needed. > + * > + * @new_cpu is ignored: the IRQ is re-affined to ctx->callee_cpus which > + * the core callee layer keeps in sync (dying CPU already removed before > + * this callback is invoked, so the mask is always accurate). > + */ > +static int arm_update_affinity(struct mbedtee_device *mbedtee, > + unsigned int new_cpu) > +{ > + struct mbedtee_rpc_transport_ctx *tctx = &mbedtee->transport; > + > + if (tctx->rpc_notify_virq > 0 && > + irq_set_affinity(tctx->rpc_notify_virq, &mbedtee->t2r.callee_cpus)) > + dev_warn(mbedtee->dev, "irq_set_affinity failed\n"); > + > + return 0; > +} > + > +static const struct rpc_transport_ops arm_transport_ops = { > + .init = arm_transport_init, > + .uninit = arm_transport_uninit, > + .update_affinity = arm_update_affinity, > +}; > + > +const struct rpc_transport_ops *mbedtee_get_rpc_transport_ops(void) > +{ > + return &arm_transport_ops; > +} > diff --git a/drivers/tee/mbedtee/rpc_callee_riscv.c b/drivers/tee/mbedtee/rpc_callee_riscv.c > new file mode 100644 > index 0000000..7ac0f8e > --- /dev/null > +++ b/drivers/tee/mbedtee/rpc_callee_riscv.c > @@ -0,0 +1,203 @@ > +// SPDX-License-Identifier: GPL-2.0-only > +/* > + * Copyright (c) 2025 Xing Loong <[email protected]> > + * RISC-V MSI transport for the TEE->REE RPC callee. > + * > + * On RISC-V the TEE wakes the REE by writing a Message Signalled Interrupt > + * into the target hart's IMSIC (Incoming MSI Controller). This file allocates > + * one platform MSI from the DT-provided MSI parent and registers the hard IRQ > + * handler supplied by the core callee layer. > + * > + * CPU hotplug: the common mbedtee_cpu_offline() callback in rpc_callee.c > + * detects when the callee CPU goes offline and calls riscv_update_affinity() > + * to migrate the MSI to another online CPU. > + */ > +#include <linux/interrupt.h> > +#include <linux/msi.h> > +#include <linux/of_irq.h> > +#include <linux/smp.h> > + > +#include "mbedtee_drv.h" > + > +static int mbedtee_ring_set_hartid(struct mbedtee_device *mbedtee, > + struct rpc_ringbuf *ring, > + unsigned int cpu) > +{ > + long hartid = cpuid_to_hartid_map(cpu); > + > + if (hartid < 0) { > + dev_err(mbedtee->dev, "CPU%u has no valid hart-id\n", cpu); > + return -ERANGE; > + } > + > + WRITE_ONCE(ring->callee_hartid, (u32)hartid); > + return 0; > +} > + > +static void mbedtee_rpc_write_msi_msg(struct msi_desc *desc_msi, > + struct msi_msg *msg) > +{ > + struct mbedtee_device *mbedtee = dev_get_drvdata(desc_msi->dev); > + > + if (mbedtee) > + mbedtee->transport.rpc_msi_msg = *msg; > +} > + > +static int riscv_transport_init(struct mbedtee_device *mbedtee, > + struct rpc_ringbuf *ring, irq_handler_t handler) > +{ > + struct mbedtee_t2r_ctx *ctx = &mbedtee->t2r; > + const struct cpumask *eff; > + struct mbedtee_rpc_transport_ctx *tctx = &mbedtee->transport; > + unsigned int cpu; > + struct device *dev = mbedtee->dev; > + int virq; > + int ret; > + > + /* > + * Set a bootstrap callee_hartid (hart 0) so the TEE can poll > + * even if MSI setup fails below. > + */ > + ret = mbedtee_ring_set_hartid(mbedtee, ring, 0); > + if (ret) > + return ret; > + > + dev_set_drvdata(dev, mbedtee); > + > + /* > + * The MSI domain of an OF device is resolved only once, when the > + * platform device is created (of_msi_configure() in of/platform.c). > + * mbedtee's MSI parent is the IMSIC, whose MSI domain is created > + * later, when the IMSIC builtin_platform_driver probes -- typically > + * after this device has already been created with a NULL msi.domain. > + * > + * Deferred probing alone cannot recover from this: the driver core > + * re-runs probe() but never re-runs of_msi_configure(), so the stale > + * NULL domain would persist across every retry. Re-resolve it here > + * and defer until the IMSIC driver is up. This mirrors the RISC-V > + * APLIC/IOMMU platform-MSI drivers. > + */ > + if (!dev_get_msi_domain(dev)) { > + of_msi_configure(dev, dev->of_node); > + > + if (!dev_get_msi_domain(dev)) > + return -EPROBE_DEFER; > + } > + > + ret = platform_device_msi_init_and_alloc_irqs(dev, 1, > + mbedtee_rpc_write_msi_msg); > + if (ret) { > + dev_err(mbedtee->dev, "MSI alloc failed: %d\n", ret); > + return ret; > + } > + > + virq = msi_get_virq(dev, 0); > + if (virq <= 0) { > + dev_err(mbedtee->dev, "no MSI virq\n"); > + ret = -ENOENT; > + goto err_msi; > + } > + > + ret = request_irq(virq, handler, 0, "mbedtee-rpc", mbedtee); > + if (ret) { > + dev_err(mbedtee->dev, "request_irq %d failed: %d\n", virq, ret); > + goto err_msi; > + } > + > + eff = irq_get_effective_affinity_mask(virq); > + cpu = eff ? cpumask_first(eff) : 0; > + > + dev_dbg(mbedtee->dev, "MSI addr 0x%x%08x data %d virq %d hart %ld\n", > + tctx->rpc_msi_msg.address_hi, tctx->rpc_msi_msg.address_lo, > + tctx->rpc_msi_msg.data, virq, cpuid_to_hartid_map(cpu)); > + > + /* Inform TEE of the IMSIC identity used for T2R notifications. */ > + ret = mbedtee_ring_set_hartid(mbedtee, ring, cpu); > + if (ret) > + goto err_irq; > + /* Ensure callee_hartid is visible before advertising MSI interrupt ID. */ > + smp_store_release(&ring->callee_imsic_id, tctx->rpc_msi_msg.data); > + > + /* Track the active callee CPU for hotplug migration. */ > + ctx->callee_virq = virq; > + cpumask_set_cpu(cpu, &ctx->callee_cpus); > + > + return 0; > + > +err_irq: > + free_irq(virq, mbedtee); > + > +err_msi: > + platform_device_msi_free_irqs_all(dev); > + return ret; > +} > + > +static void riscv_transport_uninit(struct mbedtee_device *mbedtee) > +{ > + struct mbedtee_t2r_ctx *ctx = &mbedtee->t2r; > + int virq; > + > + virq = ctx->callee_virq; > + ctx->callee_virq = 0; > + if (virq > 0) > + free_irq(virq, mbedtee); > + > + platform_device_msi_free_irqs_all(mbedtee->dev); > +} > + > +/* > + * Migrate the IMSIC MSI delivery to @new_cpu. > + * > + * Ordering guarantee for the TEE reader: > + * 1. Clear callee_imsic_id to 0 -- TEE skips IMSIC path while we migrate. > + * 2. Call irq_set_affinity() -- triggers mbedtee_rpc_write_msi_msg() which > + * updates tctx->rpc_msi_msg with the new IMSIC identity. > + * 3. Update callee_hartid (WRITE_ONCE inside mbedtee_ring_set_hartid()); > + * no explicit barrier needed here since the release store on > + * callee_imsic_id in step 4 provides the ordering. > + * 4. Publish new callee_imsic_id with smp_store_release -- TEE's > + * smp_load_acquire on callee_imsic_id pairs with this store. > + */ > +static int riscv_update_affinity(struct mbedtee_device *mbedtee, > + unsigned int new_cpu) > +{ > + struct mbedtee_t2r_ctx *ctx = &mbedtee->t2r; > + struct mbedtee_rpc_transport_ctx *tctx = &mbedtee->transport; > + struct rpc_ringbuf *ring = ctx->t2r_ring; > + int virq = ctx->callee_virq; > + > + if (!virq) > + return -ENODEV; > + > + /* Step 1: prevent TEE from using the stale IMSIC identity. */ > + smp_store_release(&ring->callee_imsic_id, 0); > + > + /* Step 2: retarget the MSI; write_msi_msg callback updates rpc_msi_msg. */ > + if (irq_set_affinity(virq, cpumask_of(new_cpu))) { > + dev_warn(mbedtee->dev, "irq_set_affinity to CPU%u failed\n", > + new_cpu); > + return -EIO; > + } > + > + /* Step 3: update callee_hartid. */ > + if (mbedtee_ring_set_hartid(mbedtee, ring, new_cpu)) > + return -ERANGE; > + > + /* Step 4: publish new IMSIC id with release barrier. */ > + smp_store_release(&ring->callee_imsic_id, tctx->rpc_msi_msg.data); > + > + dev_dbg(mbedtee->dev, "T2R callee migrated to CPU%u (hart %ld)\n", > + new_cpu, cpuid_to_hartid_map(new_cpu)); > + return 0; > +} > + > +static const struct rpc_transport_ops riscv_transport_ops = { > + .init = riscv_transport_init, > + .uninit = riscv_transport_uninit, > + .update_affinity = riscv_update_affinity, > +}; > + > +const struct rpc_transport_ops *mbedtee_get_rpc_transport_ops(void) > +{ > + return &riscv_transport_ops; > +} > diff --git a/drivers/tee/mbedtee/rpc_caller.c b/drivers/tee/mbedtee/rpc_caller.c > new file mode 100644 > index 0000000..6aeb07c > --- /dev/null > +++ b/drivers/tee/mbedtee/rpc_caller.c > @@ -0,0 +1,697 @@ > +// SPDX-License-Identifier: GPL-2.0-only > +/* > + * Copyright (c) 2020 Xing Loong <[email protected]> > + * REE->TEE session management and yield-call interface. > + */ > +#include <linux/mm.h> > +#include <linux/errno.h> > +#include <linux/slab.h> > +#include <linux/types.h> > +#include <linux/delay.h> > +#include <linux/device.h> > +#include <linux/refcount.h> > +#include "mbedtee_drv.h" > + > +#define MBEDTEE_RPC_CALL_ACTIVE 0 > +#define MBEDTEE_RPC_CALL_DONE 1 > +#define MBEDTEE_RPC_CALL_INTERRUPTED 2 > +#define MBEDTEE_RPC_CALL_ABORTED 3 > + > +static void mbedtee_rpc_call_put(struct mbedtee_rpc_call *call) > +{ > + if (refcount_dec_and_test(&call->refs)) > + kfree(call); > +} > + > +int mbedtee_rpc_call_alloc(struct mbedtee_device *mbedtee, > + size_t payload_size, struct mbedtee_rpc_call **call) > +{ > + struct mbedtee_rpc_call *rcall; > + unsigned long rpc_index; > + int ret; > + > + if (sizeof(*rcall) + payload_size > PAGE_SIZE) > + return -E2BIG; > + > + rcall = kzalloc(sizeof(*rcall) + payload_size, GFP_KERNEL); > + if (!rcall) > + return -ENOMEM; > + > + init_completion(&rcall->c); > + rcall->state = MBEDTEE_RPC_CALL_ACTIVE; > + /* One reference for the caller, one for the XArray entry. */ > + refcount_set(&rcall->refs, 2); > + rcall->rpc.size = payload_size; > + rcall->rpc.ret = -EOPNOTSUPP; > + rcall->rpc_phys = mbedtee_virt_to_phys(&rcall->rpc); > + > + do { > + rpc_index = atomic_long_inc_return(&mbedtee->rpc_call_seq); > + } while (rpc_index == 0); > + > + mutex_lock(&mbedtee->rpc_calls_mutex); > + if (mbedtee->stopping) { > + mutex_unlock(&mbedtee->rpc_calls_mutex); > + kfree(rcall); > + return -ESHUTDOWN; > + } > + ret = xa_insert_irq(&mbedtee->rpc_calls, rpc_index, rcall, GFP_KERNEL); > + mutex_unlock(&mbedtee->rpc_calls_mutex); > + if (ret != 0) { > + kfree(rcall); > + return ret; > + } > + > + rcall->rpc.waiter_id = rpc_index; > + *call = rcall; > + > + return 0; > +} > + > +void mbedtee_rpc_call_free(struct mbedtee_device *mbedtee, > + struct mbedtee_rpc_call *call) > +{ > + unsigned long flags; > + unsigned long rpc_index; > + bool drop_remote = false; > + > + if (!call) > + return; > + > + rpc_index = (unsigned long)call->rpc.waiter_id; > + > + /* > + * Interrupted calls remain reachable by the TEE, while aborted calls > + * retain a quarantined remote reference. In both cases only release the > + * caller reference here. > + */ > + xa_lock_irqsave(&mbedtee->rpc_calls, flags); > + if (call->state != MBEDTEE_RPC_CALL_INTERRUPTED && > + call->state != MBEDTEE_RPC_CALL_ABORTED && > + xa_load(&mbedtee->rpc_calls, rpc_index) == call) { > + __xa_erase(&mbedtee->rpc_calls, rpc_index); > + drop_remote = true; > + } > + xa_unlock_irqrestore(&mbedtee->rpc_calls, flags); > + > + if (drop_remote) > + mbedtee_rpc_call_put(call); > + > + /* Drop the caller reference. */ > + mbedtee_rpc_call_put(call); > +} > + > +/* > + * Abort all outstanding calls at teardown: erase each entry and wake its > + * waiter so that no workqueue item can remain blocked in a completion > + * wait when destroy_workqueue() runs. > + */ > +void mbedtee_rpc_calls_abort(struct mbedtee_device *mbedtee) > +{ > + struct mbedtee_rpc_call *call; > + unsigned long flags; > + unsigned long idx; > + > + mutex_lock(&mbedtee->rpc_calls_mutex); > + WRITE_ONCE(mbedtee->stopping, true); > + /* > + * Hold xa_lock across the whole walk so a concurrent > + * mbedtee_rpc_complete_call() cannot erase entries mid-iteration. > + * xa_for_each() only takes RCU internally, so it does not recurse > + * on xa_lock. > + */ > + xa_lock_irqsave(&mbedtee->rpc_calls, flags); > + xa_for_each(&mbedtee->rpc_calls, idx, call) { > + if (xchg(&call->state, MBEDTEE_RPC_CALL_ABORTED) == > + MBEDTEE_RPC_CALL_ACTIVE) > + complete(&call->c); > + } > + xa_unlock_irqrestore(&mbedtee->rpc_calls, flags); > + mutex_unlock(&mbedtee->rpc_calls_mutex); > +} > + > +/* > + * Release the XArray at teardown. Any entries left here are calls the TEE > + * never completed; it may still hold their rpc_phys, so we deliberately > + * leak (quarantine) them rather than risk a cross-world use-after-free. > + */ > +void mbedtee_rpc_calls_release(struct mbedtee_device *mbedtee) > +{ > + struct mbedtee_rpc_call *call; > + unsigned long idx; > + unsigned int quarantined = 0; > + > + xa_for_each(&mbedtee->rpc_calls, idx, call) { > + xa_erase(&mbedtee->rpc_calls, idx); > + /* No quiesce ACK exists, so retain the remote reference. */ > + quarantined++; > + } > + xa_destroy(&mbedtee->rpc_calls); > + mutex_destroy(&mbedtee->rpc_calls_mutex); > + > + if (quarantined) > + dev_warn(mbedtee->dev, "quarantined %u RPC calls at release\n", > + quarantined); > +} > + > +void mbedtee_rpc_complete_call(struct mbedtee_device *mbedtee, > + u64 waiter_id) > +{ > + struct mbedtee_rpc_call *call; > + unsigned long rpc_index; > + unsigned long flags; > + unsigned int state; > + > + rpc_index = (unsigned long)waiter_id; > + if ((u64)rpc_index != waiter_id) > + return; > + > + xa_lock_irqsave(&mbedtee->rpc_calls, flags); > + call = xa_load(&mbedtee->rpc_calls, rpc_index); > + if (!call) { > + xa_unlock_irqrestore(&mbedtee->rpc_calls, flags); > + return; > + } > + > + state = cmpxchg(&call->state, MBEDTEE_RPC_CALL_ACTIVE, > + MBEDTEE_RPC_CALL_DONE); > + __xa_erase(&mbedtee->rpc_calls, rpc_index); > + xa_unlock_irqrestore(&mbedtee->rpc_calls, flags); > + > + /* > + * Wake the waiter before dropping the XArray reference: a killed > + * waiter that found the entry erased may release its own reference > + * concurrently, so the put must be the last touch of the call. > + */ > + if (state == MBEDTEE_RPC_CALL_ACTIVE) > + complete(&call->c); > + mbedtee_rpc_call_put(call); > +} > + > +long mbedtee_rpc_wait_for_completion(struct mbedtee_device *mbedtee, > + struct mbedtee_rpc_call *call, bool killable) > +{ > + long ret; > + unsigned int state; > + unsigned long flags; > + unsigned long rpc_index; > + > + if (!killable) { > + wait_for_completion(&call->c); > + return READ_ONCE(call->state) == MBEDTEE_RPC_CALL_ABORTED ? > + -ESHUTDOWN : 0; > + } > + > + rpc_index = (unsigned long)call->rpc.waiter_id; > + > + ret = wait_for_completion_killable(&call->c); > + if (ret == 0) > + return READ_ONCE(call->state) == MBEDTEE_RPC_CALL_ABORTED ? > + -ESHUTDOWN : 0; > + > + /* > + * Synchronize with mbedtee_rpc_complete_call() via the > + * xa_lock. If complete_call already consumed this entry > + * before we acquired the lock, the entry is already gone > + * from the xarray and the TEE already completed (or freed) > + * the call -- treat as success since the RPC is finished. > + * If the entry is still present, we hold the lock so > + * complete_call cannot race with our state transition. > + */ > + xa_lock_irqsave(&mbedtee->rpc_calls, flags); > + if (xa_load(&mbedtee->rpc_calls, rpc_index) != call) { > + state = READ_ONCE(call->state); > + xa_unlock_irqrestore(&mbedtee->rpc_calls, flags); > + return state == MBEDTEE_RPC_CALL_ABORTED ? -ESHUTDOWN : 0; > + } > + > + state = cmpxchg(&call->state, MBEDTEE_RPC_CALL_ACTIVE, > + MBEDTEE_RPC_CALL_INTERRUPTED); > + if (state == MBEDTEE_RPC_CALL_DONE) { > + xa_unlock_irqrestore(&mbedtee->rpc_calls, flags); > + return 0; > + } > + > + if (state == MBEDTEE_RPC_CALL_ACTIVE) { > + /* Publish interruption state before the TEE waiter reads it. */ > + smp_store_release(&call->rpc.interrupted, true); > + xa_unlock_irqrestore(&mbedtee->rpc_calls, flags); > + return ret; > + } > + > + xa_unlock_irqrestore(&mbedtee->rpc_calls, flags); > + return state == MBEDTEE_RPC_CALL_ABORTED ? -ESHUTDOWN : ret; > +} > + > +/* > + * Non-interruptible wait bounded by @timeout: on expiry the call is marked > + * INTERRUPTED (remote ref retained) and -ETIMEDOUT is returned, so a hung > + * TEE cannot block the caller forever. > + */ > +long mbedtee_rpc_wait_for_completion_timeout(struct mbedtee_device *mbedtee, > + struct mbedtee_rpc_call *call, > + unsigned long timeout) > +{ > + unsigned long flags; > + unsigned long rpc_index; > + unsigned int state; > + > + if (wait_for_completion_timeout(&call->c, timeout)) > + return READ_ONCE(call->state) == MBEDTEE_RPC_CALL_ABORTED ? > + -ESHUTDOWN : 0; > + > + rpc_index = (unsigned long)call->rpc.waiter_id; > + xa_lock_irqsave(&mbedtee->rpc_calls, flags); > + if (xa_load(&mbedtee->rpc_calls, rpc_index) != call) { > + state = READ_ONCE(call->state); > + xa_unlock_irqrestore(&mbedtee->rpc_calls, flags); > + return state == MBEDTEE_RPC_CALL_ABORTED ? -ESHUTDOWN : 0; > + } > + > + state = cmpxchg(&call->state, MBEDTEE_RPC_CALL_ACTIVE, > + MBEDTEE_RPC_CALL_INTERRUPTED); > + if (state == MBEDTEE_RPC_CALL_ACTIVE) > + /* Publish interruption before the TEE observes the request. */ > + smp_store_release(&call->rpc.interrupted, true); > + xa_unlock_irqrestore(&mbedtee->rpc_calls, flags); > + > + if (state == MBEDTEE_RPC_CALL_ACTIVE) > + return -ETIMEDOUT; > + > + return state == MBEDTEE_RPC_CALL_ABORTED ? -ESHUTDOWN : 0; > +} > + > +/* Local helpers for encoding/decoding the 4-bit-per-parameter type field. */ > +static inline u32 mbedtee_param_type_get(u32 types, unsigned int idx) > +{ > + return (types >> (idx * 4)) & 0xF; > +} > + > +static inline u32 mbedtee_param_type_set(u32 type, unsigned int idx) > +{ > + return (type & 0xF) << (idx * 4); > +} > + > +static int mbedtee_param_decode(struct tee_param *params, > + size_t num_params, const struct rpc_param *rp) > +{ > + size_t n; > + > + if (num_params > ARRAY_SIZE(rp->params)) > + return -EINVAL; > + > + for (n = 0; n < num_params; n++) { > + struct tee_param *p = params + n; > + const union rpc_tee_param *rtp = rp->params + n; > + u32 attr = mbedtee_param_type_get(rp->params_type, n); > + > + /* > + * The TEE must not change the parameter type: the generic > + * TEE layer releases memrefs based on the attributes the > + * client supplied, so a type change here would leak the > + * shared memory reference. > + */ > + if (attr != p->attr) > + return -EINVAL; > + > + switch (attr) { > + case TEE_IOCTL_PARAM_ATTR_TYPE_NONE: > + memset(&p->u, 0, sizeof(p->u)); > + break; > + case TEE_IOCTL_PARAM_ATTR_TYPE_VALUE_INPUT: > + case TEE_IOCTL_PARAM_ATTR_TYPE_VALUE_OUTPUT: > + case TEE_IOCTL_PARAM_ATTR_TYPE_VALUE_INOUT: > + p->u.value.a = rtp->value.a; > + p->u.value.b = rtp->value.b; > + break; > + case TEE_IOCTL_PARAM_ATTR_TYPE_MEMREF_INPUT: > + case TEE_IOCTL_PARAM_ATTR_TYPE_MEMREF_OUTPUT: > + case TEE_IOCTL_PARAM_ATTR_TYPE_MEMREF_INOUT: > + p->u.memref.size = rtp->memref.size; > + break; > + default: > + return -EINVAL; > + } > + } > + return 0; > +} > + > +static int mbedtee_param_encode(struct rpc_param *rp, > + size_t num_params, const struct tee_param *params) > +{ > + size_t n; > + > + if (num_params > ARRAY_SIZE(rp->params)) > + return -EINVAL; > + > + rp->params_type = TEE_IOCTL_PARAM_ATTR_TYPE_NONE; > + > + for (n = 0; n < num_params; n++) { > + const struct tee_param *p = params + n; > + union rpc_tee_param *rtp = rp->params + n; > + > + rp->params_type |= mbedtee_param_type_set(p->attr, n); > + > + switch (p->attr) { > + case TEE_IOCTL_PARAM_ATTR_TYPE_NONE: > + memset(rtp, 0, sizeof(*rtp)); > + break; > + case TEE_IOCTL_PARAM_ATTR_TYPE_VALUE_INPUT: > + case TEE_IOCTL_PARAM_ATTR_TYPE_VALUE_OUTPUT: > + case TEE_IOCTL_PARAM_ATTR_TYPE_VALUE_INOUT: > + rtp->value.a = p->u.value.a; > + rtp->value.b = p->u.value.b; > + break; > + case TEE_IOCTL_PARAM_ATTR_TYPE_MEMREF_INPUT: > + case TEE_IOCTL_PARAM_ATTR_TYPE_MEMREF_OUTPUT: > + case TEE_IOCTL_PARAM_ATTR_TYPE_MEMREF_INOUT: > + if (p->u.memref.shm) > + rtp->memref.id = p->u.memref.shm->sec_world_id; > + else > + rtp->memref.id = 0; /* invalid-id @ mbedtee */ > + rtp->memref.size = p->u.memref.size; > + rtp->memref.offset = p->u.memref.shm_offs; > + break; > + default: > + return -EINVAL; > + } > + } > + > + return 0; > +} > + > +/* > + * cancel_req() and shm_{register,unregister} still return Linux errno to the > + * kernel TEE core. Preserve any local transport failure in @ret; otherwise > + * fold a non-success TEE-side GP status into the generic -EIO expected by > + * these internal-only hooks. > + */ > +static int mbedtee_rpc_gp_ret_to_errno(int ret, s32 gp_ret) > +{ > + if (ret) > + return ret; > + > + return gp_ret == TEEC_SUCCESS ? 0 : -EIO; > +} > + > +struct mbedtee_session * > +mbedtee_find_session_locked(struct mbedtee_context_data *ctxdata, > + u32 session_id) > +{ > + struct mbedtee_session *sess; > + > + list_for_each_entry(sess, &ctxdata->sess_list, list_node) > + if (sess->session_id == session_id) > + return sess; > + > + return NULL; > +} > + > +int mbedtee_open_session(struct tee_context *ctx, > + struct tee_ioctl_open_session_arg *arg, > + struct tee_param *param) > +{ > + int ret; > + u32 session_id = 0; > + struct mbedtee_context_data *ctxdata = ctx->data; > + struct mbedtee_device *mbedtee = tee_get_drvdata(ctx->teedev); > + struct mbedtee_rpc_call *call; > + struct rpc_param *rp; > + struct mbedtee_session *sess; > + > + sess = kzalloc_obj(*sess, GFP_KERNEL); > + if (!sess) > + return -ENOMEM; > + > + ret = mbedtee_rpc_call_alloc(mbedtee, sizeof(*rp), &call); > + if (ret != 0) { > + kfree(sess); > + return ret; > + } > + > + rp = (struct rpc_param *)call->rpc.data; > + > + ret = mbedtee_param_encode(rp, arg->num_params, param); > + if (ret) { > + kfree(sess); > + mbedtee_rpc_call_free(mbedtee, call); > + return ret; > + } > + > + memcpy(rp->uuid, arg->uuid, sizeof(arg->uuid)); > + memcpy(rp->clnt_uuid, arg->clnt_uuid, sizeof(arg->clnt_uuid)); > + rp->ret_origin = TEEC_ORIGIN_COMMS; > + > + ret = mbedtee_rpc_yieldcall(mbedtee, MBEDTEE_RPC_OPEN_SESSION, call, false); > + > + dev_dbg(mbedtee->dev, "open session ret %d gp_ret %d\n", ret, > + call->rpc.ret); > + > + if (ret == 0 && call->rpc.ret == TEEC_SUCCESS) { > + session_id = rp->session_id; > + sess->session_id = session_id; > + mutex_lock(&ctxdata->mutex); > + list_add(&sess->list_node, &ctxdata->sess_list); > + mutex_unlock(&ctxdata->mutex); > + } else { > + kfree(sess); > + } > + > + if (ret != 0) { > + arg->ret = TEEC_ERROR_COMMUNICATION; > + arg->ret_origin = TEEC_ORIGIN_COMMS; > + } else if (mbedtee_param_decode(param, arg->num_params, rp)) { > + arg->ret = TEEC_ERROR_COMMUNICATION; > + arg->ret_origin = TEEC_ORIGIN_COMMS; > + if (call->rpc.ret == TEEC_SUCCESS) > + mbedtee_close_session(ctx, session_id); > + } else { > + arg->session = rp->session_id; > + arg->ret = (u32)call->rpc.ret; > + arg->ret_origin = rp->ret_origin; > + } > + > + mbedtee_rpc_call_free(mbedtee, call); > + return 0; > +} > + > +int mbedtee_invoke_func(struct tee_context *ctx, > + struct tee_ioctl_invoke_arg *arg, > + struct tee_param *param) > +{ > + int ret; > + struct mbedtee_context_data *ctxdata = ctx->data; > + struct mbedtee_device *mbedtee = tee_get_drvdata(ctx->teedev); > + struct mbedtee_rpc_call *call; > + struct rpc_param *rp; > + struct mbedtee_session *sess; > + > + mutex_lock(&ctxdata->mutex); > + sess = mbedtee_find_session_locked(ctxdata, arg->session); > + if (!sess || sess->closing) { > + mutex_unlock(&ctxdata->mutex); > + return -EINVAL; > + } > + mutex_unlock(&ctxdata->mutex); > + > + ret = mbedtee_rpc_call_alloc(mbedtee, sizeof(*rp), &call); > + if (ret != 0) > + return ret; > + > + rp = (struct rpc_param *)call->rpc.data; > + > + rp->session_id = arg->session; > + rp->cmd_id = arg->func; > + rp->ret_origin = TEEC_ORIGIN_COMMS; > + > + ret = mbedtee_param_encode(rp, arg->num_params, param); > + if (ret) { > + mbedtee_rpc_call_free(mbedtee, call); > + return ret; > + } > + > + ret = mbedtee_rpc_yieldcall(mbedtee, MBEDTEE_RPC_INVOKE_SESSION, call, true); > + > + if (ret != 0) { > + /* > + * On -ERESTARTSYS/-ETIMEDOUT/-ESHUTDOWN the call is INTERRUPTED > + * or ABORTED, so free() drops only the caller reference; the > + * XArray keeps the remote reference until the TEE completes it. > + */ > + arg->ret = TEEC_ERROR_COMMUNICATION; > + arg->ret_origin = TEEC_ORIGIN_COMMS; > + mbedtee_rpc_call_free(mbedtee, call); > + return 0; > + } > + > + dev_dbg(mbedtee->dev, "invoke session ret %d gp_ret %d\n", ret, > + call->rpc.ret); > + > + if (mbedtee_param_decode(param, arg->num_params, rp)) { > + arg->ret = TEEC_ERROR_COMMUNICATION; > + arg->ret_origin = TEEC_ORIGIN_COMMS; > + } else { > + arg->ret = (u32)call->rpc.ret; > + arg->ret_origin = rp->ret_origin; > + } > + > + mbedtee_rpc_call_free(mbedtee, call); > + return 0; > +} > + > +int mbedtee_close_session(struct tee_context *ctx, u32 session) > +{ > + int ret; > + struct mbedtee_context_data *ctxdata = ctx->data; > + struct mbedtee_device *mbedtee = tee_get_drvdata(ctx->teedev); > + struct mbedtee_rpc_call *call; > + struct rpc_param *rp; > + struct mbedtee_session *sess; > + bool remove; > + > + mutex_lock(&ctxdata->mutex); > + sess = mbedtee_find_session_locked(ctxdata, session); > + if (!sess) { > + mutex_unlock(&ctxdata->mutex); > + return -EINVAL; > + } > + if (sess->closing) { > + mutex_unlock(&ctxdata->mutex); > + return -EBUSY; > + } > + sess->closing = true; > + mutex_unlock(&ctxdata->mutex); > + > + ret = mbedtee_rpc_call_alloc(mbedtee, sizeof(*rp), &call); > + if (ret != 0) > + goto out_session; > + > + rp = (struct rpc_param *)call->rpc.data; > + rp->session_id = session; > + > + ret = mbedtee_rpc_yieldcall(mbedtee, MBEDTEE_RPC_CLOSE_SESSION, call, false); > + > + mbedtee_rpc_call_free(mbedtee, call); > + > +out_session: > + remove = ret == 0 || READ_ONCE(mbedtee->stopping); > + mutex_lock(&ctxdata->mutex); > + sess = mbedtee_find_session_locked(ctxdata, session); > + if (sess) { > + if (remove) > + list_del(&sess->list_node); > + else > + sess->closing = false; > + } > + mutex_unlock(&ctxdata->mutex); > + if (remove) > + kfree(sess); > + > + return ret; > +} > + > +int mbedtee_cancel_req(struct tee_context *ctx, > + u32 cancel_id, u32 session) > +{ > + int ret; > + struct mbedtee_context_data *ctxdata = ctx->data; > + struct mbedtee_device *mbedtee = tee_get_drvdata(ctx->teedev); > + struct mbedtee_rpc_call *call; > + struct rpc_cancel_req *cancel; > + struct mbedtee_session *sess; > + > + mutex_lock(&ctxdata->mutex); > + sess = mbedtee_find_session_locked(ctxdata, session); > + if (!sess || sess->closing) { > + mutex_unlock(&ctxdata->mutex); > + return -EINVAL; > + } > + mutex_unlock(&ctxdata->mutex); > + > + ret = mbedtee_rpc_call_alloc(mbedtee, sizeof(*cancel), &call); > + if (ret != 0) > + return ret; > + > + cancel = (struct rpc_cancel_req *)call->rpc.data; > + cancel->session_id = session; > + cancel->cancel_id = cancel_id; > + > + ret = mbedtee_rpc_yieldcall(mbedtee, MBEDTEE_RPC_CANCEL, call, false); > + ret = mbedtee_rpc_gp_ret_to_errno(ret, call->rpc.ret); > + > + mbedtee_rpc_call_free(mbedtee, call); > + > + return ret; > +} > + > +int mbedtee_shm_register(struct tee_context *ctx, struct tee_shm *shm, > + struct page **pages, size_t nr_pages, unsigned long start) > +{ > + int ret; > + size_t i, j; > + u64 *pagearray; > + struct mbedtee_device *mbedtee = tee_get_drvdata(ctx->teedev); > + struct mbedtee_rpc_call *call; > + struct rpc_memref *memref; > + /* Multiple: number of MBEDTEE_PAGE_SIZE pages per Linux page. */ > + const size_t multiple = PAGE_SIZE / MBEDTEE_PAGE_SIZE; > + > + BUILD_BUG_ON(PAGE_SIZE < MBEDTEE_PAGE_SIZE); > + > + ret = mbedtee_rpc_call_alloc(mbedtee, sizeof(*memref), &call); > + if (ret != 0) > + return ret; > + > + memref = (struct rpc_memref *)call->rpc.data; > + > + pagearray = kcalloc(nr_pages * multiple, sizeof(*pagearray), GFP_KERNEL); > + if (!pagearray) { > + ret = -ENOMEM; > + goto out; > + } > + > + for (i = 0; i < nr_pages; i++) { > + for (j = 0; j < multiple; j++) > + pagearray[i * multiple + j] = MBEDTEE_PAGE_SIZE * j + > + page_to_phys(pages[i]); > + } > + > + memref->size = tee_shm_get_size(shm); > + memref->offset = tee_shm_get_page_offset(shm); > + memref->pages = mbedtee_virt_to_phys(pagearray); > + memref->cnt = nr_pages * multiple; > + ret = mbedtee_rpc_yieldcall(mbedtee, MBEDTEE_RPC_REGISTER_SHM, call, false); > + ret = mbedtee_rpc_gp_ret_to_errno(ret, call->rpc.ret); > + > + if (ret == 0) > + shm->sec_world_id = memref->id; > + > + kfree(pagearray); > + > +out: > + mbedtee_rpc_call_free(mbedtee, call); > + return ret; > +} > + > +int mbedtee_shm_unregister(struct tee_context *ctx, > + struct tee_shm *shm) > +{ > + int ret; > + struct mbedtee_device *mbedtee = tee_get_drvdata(ctx->teedev); > + struct mbedtee_rpc_call *call; > + struct rpc_memref *memref; > + > + ret = mbedtee_rpc_call_alloc(mbedtee, sizeof(*memref), &call); > + if (ret != 0) > + return ret; > + > + memref = (struct rpc_memref *)call->rpc.data; > + memref->id = shm->sec_world_id; > + > + ret = mbedtee_rpc_yieldcall(mbedtee, MBEDTEE_RPC_UNREGISTER_SHM, call, false); > + ret = mbedtee_rpc_gp_ret_to_errno(ret, call->rpc.ret); > + > + mbedtee_rpc_call_free(mbedtee, call); > + > + return ret; > +} > diff --git a/drivers/tee/mbedtee/rpc_caller_arm.c b/drivers/tee/mbedtee/rpc_caller_arm.c > new file mode 100644 > index 0000000..520b57d > --- /dev/null > +++ b/drivers/tee/mbedtee/rpc_caller_arm.c > @@ -0,0 +1,70 @@ > +// SPDX-License-Identifier: GPL-2.0-only > +/* > + * Copyright (c) 2020 Xing Loong <[email protected]> > + * ARM RPC calls to TEE - relies on SMC > + * > + * Uses arm_smccc_smc() with SMCCC-compatible calling convention: > + * Fast call : r0/x0 = fn (bit 31 set), r1-r3/x1-x3 = args. > + * Yield call : r0/x0 = fn (bit 31 clear), r1/x1 = phys(rpc_cmd). > + * > + * ARM fast calls are synchronous SMC instructions. > + */ > +#include <linux/mm.h> > +#include <linux/errno.h> > +#include <linux/slab.h> > +#include <linux/types.h> > +#include <linux/delay.h> > +#include <linux/device.h> > +#include <linux/arm-smccc.h> > +#include "mbedtee_drv.h" > + > +long mbedtee_rpc_yieldcall(struct mbedtee_device *mbedtee, > + unsigned long fn, struct mbedtee_rpc_call *call, > + bool interruptible) > +{ > + long ret; > + struct arm_smccc_res res; > + > + if (MBEDTEE_RPC_IS_FASTCALL(fn)) > + return -EINVAL; > + > + call->rpc.id = fn; > + > + arm_smccc_smc(fn, call->rpc_phys, > + 0, 0, 0, 0, 0, 0, &res); > + ret = res.a0; > + if (ret != 0) > + return ret; > + > + return mbedtee_rpc_wait_for_completion(mbedtee, call, interruptible); > +} > + > +long mbedtee_rpc_fastcall(struct mbedtee_device *mbedtee, > + unsigned long fn, unsigned long a0, > + unsigned long a1, unsigned long a2) > +{ > + struct arm_smccc_res res; > + > + if (!MBEDTEE_RPC_IS_FASTCALL(fn)) > + return -EINVAL; > + > + arm_smccc_smc(fn, a0, a1, a2, 0, 0, 0, 0, &res); > + > + return (long)res.a0; > +} > + > +/* > + * ARM uses direct SMC; no caller-side ring buffer initialisation needed. > + */ > +int mbedtee_r2t_init(struct mbedtee_device *mbedtee) > +{ > + return 0; > +} > + > +void mbedtee_r2t_stop(struct mbedtee_device *mbedtee) > +{ > +} > + > +void mbedtee_r2t_uninit(struct mbedtee_device *mbedtee) > +{ > +} > diff --git a/drivers/tee/mbedtee/rpc_caller_riscv.c b/drivers/tee/mbedtee/rpc_caller_riscv.c > new file mode 100644 > index 0000000..98f8cd3 > --- /dev/null > +++ b/drivers/tee/mbedtee/rpc_caller_riscv.c > @@ -0,0 +1,238 @@ > +// SPDX-License-Identifier: GPL-2.0-only > +/* > + * Copyright (c) 2025 Xing Loong <[email protected]> > + * REE->TEE RPC calls for RISC-V (IMSIC) > + */ > +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt > + > +#include <linux/mm.h> > +#include <linux/errno.h> > +#include <linux/slab.h> > +#include <linux/types.h> > +#include <linux/delay.h> > +#include <linux/device.h> > +#include <linux/jiffies.h> > +#include <linux/of_address.h> > +#include <linux/io.h> > +#include <linux/sizes.h> > + > +#include "mbedtee_drv.h" > + > +/* > + * Time the writer may wait for ring space before giving up. Transient > + * fullness is common under high concurrency; permanent fullness means > + * the TEE is no longer draining and must not hang ioctls forever. > + */ > +#define MBEDTEE_R2T_FULL_TIMEOUT (10 * HZ) > +#define MBEDTEE_R2T_CALL_TIMEOUT (30 * HZ) > + > +static bool rpc_ring_enough(struct mbedtee_r2t_ctx *ctx, size_t size) > +{ > + u32 wr; > + u32 rd; > + u32 remain; > + > + wr = READ_ONCE(ctx->ring_wr); > + /* Pair with callee release store when publishing ring->rd */ > + rd = smp_load_acquire(&ctx->ring->rd); > + > + if (rd > ctx->ring_sz || wr > ctx->ring_sz) > + return false; > + > + if (rd <= wr) > + remain = ctx->ring_sz + rd - wr; > + else > + remain = rd - wr; > + > + return remain > size; > +} > + > +static void rpc_ring_write(struct mbedtee_r2t_ctx *ctx, > + void *data, size_t size) > +{ > + struct rpc_ringbuf *shm = ctx->ring; > + u32 remain; > + u32 wr = READ_ONCE(ctx->ring_wr); > + > + if (wr + size > ctx->ring_sz) { > + remain = wr + size - ctx->ring_sz; > + memcpy(&shm->mem[wr], data, size - remain); > + memcpy(&shm->mem[0], (char *)data + size - remain, remain); > + wr = remain; > + } else { > + memcpy(&shm->mem[wr], data, size); > + wr += size; > + } > + > + WRITE_ONCE(ctx->ring_wr, wr); > + /* Publish writer index after payload bytes become visible. */ > + smp_store_release(&shm->wr, wr); > +} > + > +/* > + * Wait for @size bytes of space in the R2T ring, then write @data > + * atomically under the ring lock. Returns 0 on success. > + */ > +static int r2t_ring_write_wait(struct mbedtee_r2t_ctx *ctx, > + void *data, size_t size) > +{ > + unsigned long deadline = jiffies + MBEDTEE_R2T_FULL_TIMEOUT; > + unsigned long flags; > + > + for (;;) { > + spin_lock_irqsave(&ctx->lock, flags); > + if (READ_ONCE(ctx->stopping)) { > + spin_unlock_irqrestore(&ctx->lock, flags); > + return -ESHUTDOWN; > + } > + if (rpc_ring_enough(ctx, size)) { > + rpc_ring_write(ctx, data, size); > + spin_unlock_irqrestore(&ctx->lock, flags); > + return 0; > + } > + spin_unlock_irqrestore(&ctx->lock, flags); > + > + if (time_after_eq(jiffies, deadline)) > + return -ENOSPC; > + usleep_range(1000, 2000); > + } > +} > + > +long mbedtee_rpc_yieldcall(struct mbedtee_device *mbedtee, > + unsigned long fn, struct mbedtee_rpc_call *call, > + bool interruptible) > +{ > + struct mbedtee_r2t_ctx *ctx = &mbedtee->r2t; > + u64 phys; > + int ret; > + > + if (!ctx->ring) > + return -ENXIO; > + > + if (MBEDTEE_RPC_IS_FASTCALL(fn)) > + return -EINVAL; > + > + call->rpc.id = fn; > + > + phys = call->rpc_phys; > + ret = r2t_ring_write_wait(ctx, &phys, sizeof(phys)); > + if (ret != 0) { > + dev_err_ratelimited(mbedtee->dev, "rpc ring full\n"); > + return ret; > + } > + > + if (interruptible) > + return mbedtee_rpc_wait_for_completion(mbedtee, call, true); > + > + return mbedtee_rpc_wait_for_completion_timeout(mbedtee, call, > + MBEDTEE_R2T_CALL_TIMEOUT); > +} > + > +long mbedtee_rpc_fastcall(struct mbedtee_device *mbedtee, > + unsigned long fn, unsigned long a0, > + unsigned long a1, unsigned long a2) > +{ > + struct mbedtee_r2t_ctx *ctx = &mbedtee->r2t; > + struct mbedtee_rpc_call *call; > + u64 phys; > + int ret; > + > + if (!ctx->ring) > + return -ENXIO; > + > + if (!MBEDTEE_RPC_IS_FASTCALL(fn)) > + return -EINVAL; > + > + ret = mbedtee_rpc_call_alloc(mbedtee, 3 * sizeof(u64), &call); > + if (ret != 0) > + return ret; > + > + call->rpc.id = fn; > + call->rpc.size = 3 * sizeof(u64); > + call->rpc.data[0] = a0; > + call->rpc.data[1] = a1; > + call->rpc.data[2] = a2; > + > + phys = call->rpc_phys; > + ret = r2t_ring_write_wait(ctx, &phys, sizeof(phys)); > + if (ret != 0) { > + dev_err_ratelimited(mbedtee->dev, "rpc ring full\n"); > + mbedtee_rpc_call_free(mbedtee, call); > + return ret; > + } > + > + ret = mbedtee_rpc_wait_for_completion_timeout(mbedtee, call, > + MBEDTEE_R2T_CALL_TIMEOUT); > + if (ret != 0) { > + mbedtee_rpc_call_free(mbedtee, call); > + return ret; > + } > + > + ret = call->rpc.ret; > + mbedtee_rpc_call_free(mbedtee, call); > + > + return ret; > +} > + > +int mbedtee_r2t_init(struct mbedtee_device *mbedtee) > +{ > + struct mbedtee_r2t_ctx *ctx = &mbedtee->r2t; > + struct device_node *node = mbedtee->dev->of_node; > + struct resource res; > + int ret; > + > + memset(ctx, 0, sizeof(*ctx)); > + spin_lock_init(&ctx->lock); > + > + ret = mbedtee_get_resource(node, "r2t-ring", &res); > + if (ret) > + return ret; > + > + if (resource_size(&res) <= sizeof(struct rpc_ringbuf)) { > + dev_err(mbedtee->dev, "r2t-ring too small\n"); > + return -EINVAL; > + } > + > + ctx->ring = memremap(res.start, resource_size(&res), > + MEMREMAP_WB); > + if (!ctx->ring) { > + dev_err(mbedtee->dev, "failed to map r2t ring at %pa\n", > + &res.start); > + return -ENOMEM; > + } > + ctx->ring_sz = resource_size(&res) - sizeof(struct rpc_ringbuf); > + /* Read initial writer index with acquire for coherent ring bootstrap. */ > + WRITE_ONCE(ctx->ring_wr, smp_load_acquire(&ctx->ring->wr)); > + > + dev_dbg(mbedtee->dev, "r2t-ring %pa\n", &res.start); > + > + return 0; > +} > + > +void mbedtee_r2t_stop(struct mbedtee_device *mbedtee) > +{ > + struct mbedtee_r2t_ctx *ctx = &mbedtee->r2t; > + unsigned long flags; > + > + spin_lock_irqsave(&ctx->lock, flags); > + WRITE_ONCE(ctx->stopping, true); > + spin_unlock_irqrestore(&ctx->lock, flags); > +} > + > +void mbedtee_r2t_uninit(struct mbedtee_device *mbedtee) > +{ > + struct mbedtee_r2t_ctx *ctx; > + > + if (!mbedtee) > + return; > + > + ctx = &mbedtee->r2t; > + if (ctx->ring) { > + /* r2t_init() failure lands here with no prior stop; flag first. */ > + mbedtee_r2t_stop(mbedtee); > + memunmap(ctx->ring); > + ctx->ring = NULL; > + ctx->ring_sz = 0; > + ctx->ring_wr = 0; > + } > +} > diff --git a/drivers/tee/mbedtee/shm_pool.c b/drivers/tee/mbedtee/shm_pool.c > new file mode 100644 > index 0000000..708ab24 > --- /dev/null > +++ b/drivers/tee/mbedtee/shm_pool.c > @@ -0,0 +1,105 @@ > +// SPDX-License-Identifier: GPL-2.0-only > +/* > + * Copyright (c) 2015, Linaro Limited > + * Copyright (c) 2017, EPAM Systems > + * Copyright (c) 2020 Xing Loong <[email protected]> > + */ > +#include <linux/device.h> > +#include <linux/genalloc.h> > +#include <linux/slab.h> > + > +#include "mbedtee_drv.h" > + > +static int pool_op_alloc(struct tee_shm_pool *pool, > + struct tee_shm *shm, size_t size, size_t align) > +{ > + unsigned int order = get_order(size); > + struct page *page; > + int rc = 0; > + > + page = alloc_pages(GFP_KERNEL | __GFP_ZERO, order); > + if (!page) > + return -ENOMEM; > + > + shm->kaddr = page_address(page); > + shm->paddr = page_to_phys(page); > + shm->size = PAGE_SIZE << order; > + > + if (!(shm->flags & TEE_SHM_PRIV)) { > + struct mbedtee_device *mbedtee = tee_get_drvdata(shm->ctx->teedev); > + > + if (mbedtee->yield) { > + unsigned int nr_pages = 1 << order; > + unsigned int i; > + struct page **pages; > + > + pages = kmalloc_array(nr_pages, sizeof(*pages), GFP_KERNEL); > + if (!pages) { > + rc = -ENOMEM; > + goto err_pages; > + } > + > + for (i = 0; i < nr_pages; i++) > + pages[i] = page + i; > + > + rc = mbedtee_shm_register(shm->ctx, shm, pages, > + nr_pages, > + (unsigned long)shm->kaddr); > + kfree(pages); > + if (rc) > + goto err_pages; > + } > + } > + > + return 0; > + > +err_pages: > + free_pages((unsigned long)shm->kaddr, get_order(shm->size)); > + shm->kaddr = NULL; > + return rc; > +} > + > +static void pool_op_free(struct tee_shm_pool *pool, > + struct tee_shm *shm) > +{ > + if (!(shm->flags & TEE_SHM_PRIV)) { > + struct mbedtee_device *mbedtee = tee_get_drvdata(shm->ctx->teedev); > + > + if (mbedtee->yield && shm->sec_world_id) > + mbedtee_shm_unregister(shm->ctx, shm); > + } > + > + free_pages((unsigned long)shm->kaddr, get_order(shm->size)); > + shm->kaddr = NULL; > +} > + > +static void pool_op_destroy_pool(struct tee_shm_pool *pool) > +{ > + kfree(pool); > +} > + > +static const struct tee_shm_pool_ops pool_ops = { > + .alloc = pool_op_alloc, > + .free = pool_op_free, > + .destroy_pool = pool_op_destroy_pool, > +}; > + > +/** > + * mbedtee_shm_pool_alloc_pages() - create page-based allocator pool > + * > + * This pool is used when MbedTEE supports dynamic shared memory. Command > + * buffers and similar structures are allocated from kernel's own memory. > + * > + * Return: pointer to a tee_shm_pool or ERR_PTR on failure > + */ > +struct tee_shm_pool *mbedtee_shm_pool_alloc_pages(void) > +{ > + struct tee_shm_pool *pool = kzalloc_obj(*pool, GFP_KERNEL); > + > + if (!pool) > + return ERR_PTR(-ENOMEM); > + > + pool->ops = &pool_ops; > + > + return pool; > +} > diff --git a/drivers/tee/mbedtee/shm_pool.h b/drivers/tee/mbedtee/shm_pool.h > new file mode 100644 > index 0000000..4510676 > --- /dev/null > +++ b/drivers/tee/mbedtee/shm_pool.h > @@ -0,0 +1,15 @@ > +/* SPDX-License-Identifier: GPL-2.0-only */ > +/* > + * Copyright (c) 2015, Linaro Limited > + * Copyright (c) 2016, EPAM Systems > + * Copyright (c) 2020 Xing Loong <[email protected]> > + */ > + > +#ifndef SHM_POOL_H > +#define SHM_POOL_H > + > +#include <linux/tee_drv.h> > + > +struct tee_shm_pool *mbedtee_shm_pool_alloc_pages(void); > + > +#endif > diff --git a/drivers/tee/mbedtee/supp.c b/drivers/tee/mbedtee/supp.c > new file mode 100644 > index 0000000..1408403 > --- /dev/null > +++ b/drivers/tee/mbedtee/supp.c > @@ -0,0 +1,311 @@ > +// SPDX-License-Identifier: GPL-2.0-only > +/* > + * Copyright (c) 2015, Linaro Limited > + * Copyright (c) 2020 Xing Loong <[email protected]> > + */ > +#include <linux/device.h> > +#include <linux/sched.h> > +#include <linux/slab.h> > +#include <linux/uaccess.h> > + > +#include "mbedtee_drv.h" > + > +struct mbedtee_supp_req { > + u32 func; > + > + struct list_head node; > + struct task_struct *worker; > + struct tee_context *ctx; /* owning supplicant context */ > + > + struct tee_shm *shm; > + void *data; > + size_t size; > + > + struct completion c; > +}; > + > +static int mbedtee_supp_check_recv_params(size_t num_params, > + struct tee_param *params) > +{ > + if (num_params != 1) > + return -EINVAL; > + > + if (!tee_param_is_memref(params) || !params->u.memref.shm) > + return -EINVAL; > + > + return 0; > +} > + > +void mbedtee_supp_init(struct mbedtee_supp *supp) > +{ > + memset(supp, 0, sizeof(*supp)); > + mutex_init(&supp->mutex); > + init_completion(&supp->reqs_c); > + INIT_LIST_HEAD(&supp->reqs); > + INIT_LIST_HEAD(&supp->active_reqs); > +} > + > +void mbedtee_supp_uninit(struct mbedtee_supp *supp) > +{ > + mutex_destroy(&supp->mutex); > +} > + > +static void mbedtee_supp_complete_req(struct mbedtee_supp_req *r) > +{ > + list_del_init(&r->node); > + if (r->shm) > + tee_shm_put(r->shm); > + r->shm = NULL; > + /* Tell the TEE that the request failed, not stale success data. */ > + if (r->data && r->size >= sizeof(struct supp_cmd_hdr)) > + ((struct supp_cmd_hdr *)r->data)->ret = TEEC_ERROR_COMMUNICATION; > + complete(&r->c); > +} > + > +void mbedtee_supp_release(struct mbedtee_supp *supp, struct tee_context *ctx) > +{ > + struct mbedtee_supp_req *r, *n; > + > + mutex_lock(&supp->mutex); > + > + /* > + * Cancel only requests owned by @ctx. Unclaimed requests and > + * requests owned by other supplicant contexts are left alone. > + */ > + list_for_each_entry_safe(r, n, &supp->reqs, node) { > + if (r->ctx != ctx) > + continue; > + mbedtee_supp_complete_req(r); > + } > + > + list_for_each_entry_safe(r, n, &supp->active_reqs, node) { > + if (r->ctx != ctx) > + continue; > + mbedtee_supp_complete_req(r); > + } > + > + if (supp->ctx == ctx) > + supp->ctx = NULL; > + > + mutex_unlock(&supp->mutex); > +} > + > +void mbedtee_supp_abort_all(struct mbedtee_supp *supp) > +{ > + struct mbedtee_supp_req *r, *n; > + > + mutex_lock(&supp->mutex); > + supp->shutting_down = true; > + > + list_for_each_entry_safe(r, n, &supp->reqs, node) > + mbedtee_supp_complete_req(r); > + > + list_for_each_entry_safe(r, n, &supp->active_reqs, node) > + mbedtee_supp_complete_req(r); > + > + supp->ctx = NULL; > + > + mutex_unlock(&supp->mutex); > + complete_all(&supp->reqs_c); > +} > + > +static int supp_enqueue_req(struct mbedtee_supp *supp, > + struct mbedtee_supp_req *req) > +{ > + int ret = 0; > + > + mutex_lock(&supp->mutex); > + if (supp->shutting_down) > + ret = -ESHUTDOWN; > + else > + list_add_tail(&req->node, &supp->reqs); > + mutex_unlock(&supp->mutex); > + > + return ret; > +} > + > +/* > + * Receive an RPC request from TEE and dispatch it to supplicant. > + * Blocks until the supplicant sends back the result. > + */ > +void mbedtee_supp_handler(struct mbedtee_device *mbedtee, > + u32 func, void *data, size_t size) > +{ > + struct mbedtee_supp *supp = &mbedtee->supp; > + struct mbedtee_supp_req *req = kzalloc_obj(*req, GFP_KERNEL); > + struct supp_cmd_hdr *cmd = data; > + int ret; > + > + if (!req) { > + if (cmd && size >= sizeof(*cmd)) > + cmd->ret = TEEC_ERROR_OUT_OF_MEMORY; > + return; > + } > + > + init_completion(&req->c); > + req->func = func; > + req->data = data; > + req->size = size; > + > + ret = supp_enqueue_req(supp, req); > + if (ret != 0) { > + if (cmd && size >= sizeof(*cmd)) > + cmd->ret = TEEC_ERROR_COMMUNICATION; > + kfree(req); > + return; > + } > + > + /* Wake up the supplicant daemon to handle this request. */ > + complete(&supp->reqs_c); > + > + /* > + * Wait for the supplicant to process and return the result, like > + * OP-TEE does. If the supplicant dies, mbedtee_supp_release() > + * completes the request, so this cannot block forever. > + */ > + wait_for_completion(&req->c); > + > + kfree(req); > +} > + > +/* > + * Called by supplicant to receive the next pending request. > + */ > +int mbedtee_supp_recv(struct tee_context *ctx, > + u32 *func, u32 *num_params, struct tee_param *param) > +{ > + struct tee_device *teedev = ctx->teedev; > + struct mbedtee_device *mbedtee = tee_get_drvdata(teedev); > + struct mbedtee_supp *supp = &mbedtee->supp; > + struct mbedtee_supp_req *req; > + struct mbedtee_context_data *d = ctx->data; > + struct tee_shm *shm; > + int ret; > + > + ret = mbedtee_supp_check_recv_params(*num_params, param); > + if (ret != 0) > + return ret; > + > + shm = param->u.memref.shm; > + shm = tee_shm_get_from_id(ctx, shm->id); > + tee_shm_put(param->u.memref.shm); > + if (IS_ERR(shm)) { > + param->u.memref.shm = NULL; > + return PTR_ERR(shm); > + } > + param->u.memref.shm = shm; > + > + d->is_supp_ctx = true; > + > + /* > + * Pop, own and validate the request in one critical section so that > + * a concurrent supp_abort_all() can never free the request while > + * it is being prepared here. > + */ > + while (true) { > + mutex_lock(&supp->mutex); > + if (supp->shutting_down) { > + mutex_unlock(&supp->mutex); > + ret = -ESHUTDOWN; > + goto err; > + } > + > + req = list_first_entry_or_null(&supp->reqs, > + struct mbedtee_supp_req, node); > + if (!req) { > + mutex_unlock(&supp->mutex); > + if (wait_for_completion_interruptible(&supp->reqs_c)) { > + ret = -ERESTARTSYS; > + goto err; > + } > + continue; > + } > + > + list_del_init(&req->node); > + req->worker = current; > + req->ctx = ctx; > + list_add_tail(&req->node, &supp->active_reqs); > + > + if (req->size > param->u.memref.size) { > + /* > + * The supplicant cannot grow its buffer dynamically, so > + * requeueing would retry this request forever. Complete it > + * with a transport error and keep this ioctl waiting for the > + * next request instead of terminating a supplicant worker. > + */ > + mbedtee_supp_complete_req(req); > + mutex_unlock(&supp->mutex); > + continue; > + } > + > + req->shm = shm; > + *func = req->func; > + memcpy(shm->kaddr, req->data, req->size); > + param->u.memref.size = req->size; > + mutex_unlock(&supp->mutex); > + > + return 0; > + } > + > +err: > + tee_shm_put(shm); > + param->u.memref.shm = NULL; > + return ret; > +} > + > +/* > + * Called by supplicant to send back the result of a request. > + */ > +int mbedtee_supp_send(struct tee_context *ctx, u32 ret, u32 num_params, > + struct tee_param *param) > +{ > + struct tee_device *teedev = ctx->teedev; > + struct mbedtee_device *mbedtee = tee_get_drvdata(teedev); > + struct mbedtee_supp *supp = &mbedtee->supp; > + struct mbedtee_supp_req *req; > + struct mbedtee_supp_req *_req; > + bool invalid; > + > + /* > + * Find and detach the active request belonging to this worker thread > + * and to this supplicant context. Each worker is identified by its > + * task_struct pointer, which was recorded when supp_recv() dispatched > + * the request. > + */ > + mutex_lock(&supp->mutex); > + req = NULL; > + list_for_each_entry(_req, &supp->active_reqs, node) { > + if (_req->worker == current && _req->ctx == ctx) { > + req = _req; > + list_del_init(&req->node); > + break; > + } > + } > + mutex_unlock(&supp->mutex); > + > + if (!req) > + return -ENOENT; > + > + /* > + * Complete the request even on malformed input: the worker waiting > + * in mbedtee_supp_handler() must not hang because a broken > + * supplicant sent invalid parameters. > + */ > + invalid = num_params != 1 || !tee_param_is_memref(param) || > + param->u.memref.size > req->size; > + if (invalid) { > + if (req->data && req->size >= sizeof(struct supp_cmd_hdr)) > + ((struct supp_cmd_hdr *)req->data)->ret = > + TEEC_ERROR_COMMUNICATION; > + } else { > + memcpy(req->data, req->shm->kaddr, param->u.memref.size); > + } > + if (req->shm) > + tee_shm_put(req->shm); > + req->shm = NULL; > + > + /* Wake up mbedtee_supp_handler(). */ > + complete(&req->c); > + > + return invalid ? -EINVAL : 0; > +} > diff --git a/include/uapi/linux/tee.h b/include/uapi/linux/tee.h > index 5203977..2d30296 100644 > --- a/include/uapi/linux/tee.h > +++ b/include/uapi/linux/tee.h > @@ -60,6 +60,7 @@ > #define TEE_IMPL_ID_AMDTEE 2 > #define TEE_IMPL_ID_TSTEE 3 > #define TEE_IMPL_ID_QTEE 4 > +#define TEE_IMPL_ID_MBEDTEE 5 > > /* > * OP-TEE specific capabilities > -- > 2.43.0 >