Re: dma_heap and protected memory

On 5/20/26 6:18 AM, Jens Wiklander wrote:
> Hi Amir,
> 
> On Wed, May 20, 2026 at 9:10 AM Amirreza Zarrabi
> <[email protected]> wrote:
>>
>> Hi Jens,
>>
>> On 5/20/2026 1:29 AM, Jens Wiklander wrote:
>>> Hi Amir,
>>>
>>> On Fri, May 15, 2026 at 5:10 AM Amirreza Zarrabi
>>> <[email protected]> wrote:
>>>>
>>>> Hi Jens,
>>>>
>>>> Recently protected heap support has been added for the TEE subsystem.
>>>> This allows dma-heaps to be managed by the TEE subsystem itself.
>>>>
>>>> For standard, non-protected heaps such as the system heap or CMA heap,
>>>> I am trying to understand the expected usage model. One possible workaround
>>>> appears to be to mmap the dma-buf fd in userspace, then register the
>>>> resulting userspace VA with the TEE subsystem to obtain a TEE shared-memory
>>>> object, for example:
>>>>
>>>> fd = dma_heap_alloc(...);
>>>> va = mmap(fd, ...);
>>>>
>>>> shm_fd = tee_shm_register(va, size);
>>>> tee_invoke(shm_fd);
>>>>
>>>> close(shm_fd);
>>>> munmap(va, size);
>>>> close(fd);
>>>>
>>>> This seems reasonable when the lifetime of the registered shared memory
>>>> is limited to a single invocation or callback. However, I am concerned about
>>>> cases (multiple exits in qcomtee) where the TEE side needs to retain access
>>>> to the memory after the ioctl/invocation returns.
>>>>
>>>> In that case, registering only the userspace VA does not appear
>>>> to keep the underlying dma-buf object alive. The TEE core pins the pages,
>>>> but it does not hold a dma-buf reference or attachment. If userspace later
>>>> closes the dma-buf fd and unmaps the VA, the dma-buf exporter may consider
>>>> the buffer released, while the TEE side may still have the memory registered.
>>>
>>> That's surprising. Are the pages returned to the heap also? If so,
>>> there's an issue.
>>>
>>
>> Yes, that is my understanding. For dma-heaps, after the final dma-buf
>> reference is released, the exporter is free to return the backing pages to
>> the heap.
> 
> This should be a generic problem. Surely there must be something to
> stop the exporter from returning the pages.
> 

This is a standard use-after-free issue. The above sequence would be:
allocate a buffer from the Heap, mmap it to get a VA, save the VA (either
locally or giving it to the TEE framework), then unmap and close the dma-buf,
TEE tries to use the VA. This would be an issue no matter how you allocated
the memory, don't free memory still being used.

But the core question is:

> where the TEE side needs to retain access to the memory after the
> ioctl/invocation returns.

For this case, are you talking about when the TEE needs to keep memory
past one invocation for a whole session or for even longer than that?
(such as a supplicant helper allocating memory for a long running TEE/TA
process)

For either case the ownership of the buffer must pass to the TEE from
the application, and that is something we would need the ability to
register generic DMA-BUFs, so adding this support does seem valid.

>>
>>>>
>>>> Is there a recommended way to use standard heaps, such as the system or
>>>> CMA dma-buf heaps, when the sharing lifetime is longer than a single TEE call?
>>>
>>> No, I think you're pioneering this.
>>>
>>
>> ;-).
>>
>>>>
>>>> Would it make sense to extend tee_shm_register_fd() so that it can handle
>>>> dma-buf fds from regular dma-heaps as well, not only dma-bufs managed by
>>>> the TEE subsystem? That would allow the TEE core or backend driver to keep
>>>> the dma-buf reference for the lifetime of the TEE shared-memory object.
>>>
>>> It seems like a workaround, since I assume that the purpose is to
>>> share memory between the user space process and the TEE. I think this
>>> might be a last resort or something.
>>>
>>
>> Yes, I agree.
>>
>> The specific case I am looking at is qcomtee without an FF-A backend. In
>> that configuration, qcomtee currently requires physically contiguous shared
>> memory, so registering anonymous userspace memory is not very practical.
> 
> Agreed.
> 
>>
>> Today the qcomtee shared-memory pool is implemented on top of
>> tee_dyn_shm_alloc_helper(), which allocates memory from the normal page
>> allocator. This makes larger allocations subject to the usual high-order
>> allocation / MAX_ORDER limitations.
>>
>> One option would be to move, or extend, the qcomtee shared-memory pool to use
>> a cma backed allocation path. That would address the default allocation case
>> through TEE_IOC_SHM_ALLOC (without restriction).
> 
> CMA maintainers might have a view on that.
> 
>>
>> However, I am wondering whether it is better to also allow userspace to choose
>> the source of the memory, for example when the memory must come from a specific
>> carveout or from a cma dma-heap selected by the client.
>>
>> If you feel it is not a correct direction, I totally understand considering
>> it is workaround for a limitation.
> 
> It has an advantage by moving the problem to user-space. I can't say
> for sure what I think of it before I've seen the needed changes.
> tee_ioctl_shm_register_fd() or one of the helper functions will need
> some updates to handle generic DMA-bufs from elsewhere.
> 

Goes back to the original issues that folks brought up when upstreaming
the "secure" Heaps. How do we signal to the TEE that a buffer is safe
to consume and use. We can check things like if it is contiguous easy
enough, but is it a "secure" buffer, etc.. now we end up needing to
think about all kinds of flags and signalling. (just things to think
about, not that any of this is a true blocker).

Andrew

> Cheers,
> Jens