Re: aarch64: Memory tagging modes proposal

On 18/08/26 07:03, Yury Khrustalev wrote:
> On Mon, Aug 17, 2026 at 04:14:17PM -0300, Adhemerval Zanella Netto wrote:
>>
>> On 11/08/26 10:40, Yury Khrustalev wrote:
>>> Request for feedback.
>>>
>>> This is a proposal for how MTE binary marking from [1] should co-exist with
>>> the MTE Glibc tunable (not yet implemented). In my opinion, both are necessary
>>> to cover all practical use cases and provide both flexibility of future
>>> experiments and ease of configuration for developers and end users. This
>>> explain how it all should work.
>>>
>>
>> Hi Yury,
>>
>> I did a POC to check how it would play with glibc [1], mostly to check how
>> to simplify the first MTE proposal but also to check if current ideas from
>> other system would make sense for glibc. 
>>
>>> [1]: https://github.com/ARM-software/abi-aa/blob/main/memtagabielf64/memtagabielf64.rst
>>>
>>> The set of rules below:
>>>
>>> 1) Respects binary marking when it's present.
>>> 2) Tunable cannot disable tagging, only change MTE mode.

I just realized that this can be arguably a security hazard, like downgrade a
DT_AARCH64_MEMTAG_MODE set to 'sync' to 'async'.

>>> 3) What kind of memory tagging is enabled (i.e. the scope) is solely
>>>    determined either by the binary marking (when it's present) or by
>>>    the tunable (when marking is not present).
>>> 4) You cannot enable stack and globals tagging unless corresponding
>>>    binary marking is present (it requires support by compiler and static
>>>    linker), but you can choose if you enable heap tagging for unmarked
>>>    binaries.
>>
>> Afaiu it should be *functional* safe to enable MTE even though the process itself
>> does not issue any MTE instructions.  It would incur in the performance hit
>> (kernel-side page cost, non-KSM merging, extra DRAM usage), but it would be a way
>> to make non-marked binaries to allow dlopen marked DSO.
> 
> It would be functional but the functionality that is "enabled" this way
> would not be in fact enabled, and I don't want this to happen silently.

It is not clear what your meant by 'not in fact enabled', my understanding is
in theory we can add a tunable that will essentially mimic what the 
DT_AARCH64_MEMTAG_* does for process startup (prctrl plus PROT_MTE on mmap calls).

But I think it would only make sense for dlopen DT_AARCH64_MEMTAG_{STACK,GLOBAL},
and we can also just avoid this if the process has not MTE enabled at program
startup.

> 
>>> 5) If you want heap tagging for marked binaries, the DT_AARCH64_MEMTAG_HEAP
>>>    tag must be set explicitly.
>>
>> I think different than DT_AARCH64_MEMTAG_STACK, DT_AARCH64_MEMTAG_HEAP can be
>> enabled independently whether the process has the marking or not. Afaiu it is
>> property of the malloc implementation instead of the binary (different than
>> the stack support).
> 
> Heap stacking *can* be enabled independently, I don't argue with that.
> What I am saying is that binary marking, if present, should take
> precedence. So, if out of components X, Y, and Z we have marking only
> for Y, we only enabled Y because this is what marking dictates. I think
> this provides for a clearer interface where users know what to expect.

Agreed, but the way 5. is written hints that memory tagging for malloc is *only*
enabled through the markings. But I am also not sure how do you plan to re-enable
malloc tagging.

> 
>>>
>>> Binary marking:
>>>  - DT_AARCH64_MEMTAG_MODE=0|1: selects fault mode for MTE memory tag
>>>    checking, i.e. MTE mode.
>>>  - DT_AARCH64_MEMTAG_{HEAP,STACK,GLOBALS} determines the scope of
>>>    memory tagging.
>>
>> I think it would be worth to add that MTE state should be setup at program
>> *startup* and it should not be changed during process execution (like by
>> dlopen). 
> 
> This is already covered by the MEMTAGABI document. This part that we're
> discussing here only covers the initial setup.

Yes, but I think we should make clear that the glibc implementation intend
to adhere to this. I am raising this because currently the only MTE implementation
reference Linux developers have (Android) allows dlopen DSO with MTE
when thread does not initially enable it.

> 
>> I don't think we should replicate bionic's DT_AARCH64_MEMTAG_STACK
>> support, where it enables MTE support for running threads if a DSO required
>> it.
> 
> I agree. This should be added to the GNU/Linux part of the MEMTAG ABI
> spec.
> 
>> It means that issuing dlopen on marking DSO for a process without marking
>> should return an error.
> 
> DSOs are not supposed to have DT_AARCH64_MEMTAG_{HEAP,STACK,GLOBALS}
> marking according to the spec, so if they do, it's an error like you
> suggest. I also agree that even if we decided to load such a DSO, we
> should not alter the state of the process.

Without the marking on DSO it would be *really* hard to avoid loading DSO that
*required* STACK/GLOBAL support if the binary does not enable it.  I think it
would be counter-productive to do this, since it would most like result
hard to debug failures (SIGILL or something alike during stack or global
operations).

And it is also not current practice to fail early to avoid process loading 
with undefined behavior.

> 
>>> Tunable:
>>>  - glibc.cpu.mtemode=none|auto|sync|async: selects fault mode for MTE
>>>    memory tag checking (also see below), default value is 'none'.
>>
>> Should we consider asymm mode? This is not defined by the ABI but it supported
>> by Linux and defined for ARM v8.7.
> 
> AFAIK, prctl flags don't have it, but if they do, we surely can support
> it (now or later).

It does, although indirectly:

Documentation/arch/arm64/memory-tagging-extension.rst

149 To allow a program to potentially run in the CPU's preferred tag
150 checking mode, the user program may set multiple tag check fault mode
151 bits in the ``flags`` argument to the ``prctl(PR_SET_TAGGED_ADDR_CTRL,
152 flags, 0, 0, 0)`` system call. If both synchronous and asynchronous
153 modes are requested then asymmetric mode may also be selected by the
154 kernel. If the CPU's preferred tag checking mode is in the task's set
155 of provided tag checking modes, that mode will be selected. Otherwise,
156 one of the modes in the task's mode will be selected by the kernel
157 from the task's mode set using the preference order:
158
159         1. Asynchronous
160         2. Asymmetric
161         3. Synchronous
162
163 Note that there is no way for userspace to request multiple modes and
164 also disable asymmetric mode.

We can map to some like 'glibc.cpu.mtemode=sync+async' to mimic the kernel,
but it makes tunable parsing a bit more complex and exposes how Linux
implements it instead of hardware support.

> 
>>> If both the tunable and the DT_AARCH64_MEMTAG_MODE tag are explicitly
>>> set, the tunable takes precedence (allows to override mode that is built
>>> into the binary) except when tunable is set to 'none' in which case the
>>> tunable has no effect.
>>>
>>> Tunable cannot disable MTE memory tagging when binary marking requires it,
>>> it can only change the mode.
>>>
>>> If none of DT_AARCH64_MEMTAG_{HEAP,STACK,GLOBALS} is present, the tunable,
>>> when set to a value other than 'none', will enable the "standard" set of
>>> memory tagging features (currently HEAP).
>>
>> I think this a bit confusing, where it mixes the MTE mode (auto, sync, async)
>> with a MTE support.
> 
> I don't want to add another tunable for on / off of MTE given that if
> the user requests some MTE mode other than "none" they mean that MTE
> should be enabled. Then the question is: What parts of Glibc should
> enabled MTE support? My answer to this: in the absence of marking we
> enable heap support only (but this can be extended in the future, that
> is why I called it "standard" scope); if marking is present, we enable
> components that are specified in the marking.

Yes, but which usercase 'none' really makes sense here? For the rules you
defined, tunables can not disable tagging (2.), and stack/global are only
gated through markings (4.). So 'none' here only gates whether enable
*heap* tagging for a process without marking. It is still confusing and
conflate two different MTE functionalities. 

> 
>> Also, it is not clear why we need 'auto' here.
> 
> 'auto' means use mode preferred by CPU as described in:
> https://www.kernel.org/doc/html/next/arm64/memory-tagging-extension.html

Yes, but what does it gain for the glibc usercase? If the binary has the
heap/global/stack marking the mode will either the mode marking of the kernel
default, so an implicit 'auto'.

If the binary has any marking, does it enable heap with kernel default mode?
If so, this is confusing.

Maybe it would be better to have a *different* tunable to control the MTE
support (glibc.cpu.mtesupport or something).

> 
>> The ISA required 'async' to be
>> supported, and without DT_AARCH64_MEMTAG_MODE it is the default. I think in
>> the absence of the mode marking we can just use the 'mte_tcf_preferred'
>> setting.
> 
> The idea is to allow choosing the mode via tunable. It is main purpose
> of this tunable. We need this because there are going to be use cases
> when people want to run *unmarked* binaries with heap tagging protection
> but when they also want to switch between modes depending on their
> goals. This is to support cases when we don't want or cannot rebuild a
> binary every time and just want to run it in different configurations.
> 
>>>
>>> If DT_AARCH64_MEMTAG_MODE is not present and the mode has to be inferred
>>> from the binary marking (i.e. in the absence of the tunable), the default
>>> value for the mode is 'auto'.
>>>
>>>  1) No binary marking: when none of the DT_AARCH64_MEMTAG_{HEAP,STACK,GLOBALS}
>>>     tags are present:
>>>
>>>   | Tunable | MEMTAG_MODE | Status | Mode  | Scope     |
>>>   |----------------------------------------------------|
>>>   | (unset) | (ignored)   | off    |  --   | --        |
>>>   | none    | (ignored)   | off    |  --   | --        |
>>>   | auto    | (ignored)   | on     | auto  | standard  |
>>>   | sync    | (ignored)   | on     | sync  | standard  |
>>>   | async   | (ignored)   | on     | async | standard  |
>>>>  2) When DT_AARCH64_MEMTAG_{HEAP,STACK,GLOBALS} are present (at least
>>>     one of them):
>>>
>>>   | Tunable | MEMTAG_MODE | Status | Mode  | Scope     |
>>>   |----------------------------------------------------|
>>>   | (unset) | (unset)     | on     | auto  | (marking) |
>>>   | (unset) | 0           | on     | sync  | (marking) |
>>>   | (unset) | 1           | on     | async | (marking) |
>>>   | none    | (unset)     | on     | auto  | (marking) |
>>>   | none    | 0           | on     | sync  | (marking) |
>>>   | none    | 1           | on     | async | (marking) |
>>>   | auto    | (ignored)   | on     | auto  | (marking) |
>>>   | sync    | (ignored)   | on     | sync  | (marking) |
>>>   | async   | (ignored)   | on     | async | (marking) |
>>>
>>> In the table above, '(marking)' means whatever components are flagged with
>>> the DT_AARCH64_MEMTAG_{HEAP,STACK,GLOBALS} tags and are supported by the
>>> current version of Glibc.
>>
>> It still not clear what the 'none' tunable mode improves here.
> 
> It is the default value of the tunable when it's not set explicitly.

Yes, but why do we need to expose this internal state to tunable? Which usercase
setting glibc.cpu.mtemode=none make sense?

> 
>> For 1. it
>> is ignored anyway,
> 
> For 1) it not ignored. It selects the mode that the user wants to use in
> this specific invocation of their program.

So for 1. it is gating heap tagging.  It is not clear why there is a need for 'none'
mode, since it means essentially to not enable heap tagging for a binary without markings.
It seems to be no-op in this case.

> 
>> and for 2. it is unclear which mode will be used
>> ('mte_tcf_preferred', async).
> 
> The mode that user specifies in the tunable will be used. We can support
> more modes such as asymmetric when it's supported in the prctl interface.

But for 2. it will either the binary marking mode *or*, in its absence,
the kernel default. So assuming binary has either stack/global/heap:

 1. If binary has DT_AARCH64_MEMTAG_MODE, what does 'none' provides exactly?

 2. If binary has not DT_AARCH64_MEMTAG_MODE, what is the difference for 'auto'?
> 
>>>
>>> Status 'on' means that the prctl(PR_TAGGED_ADDR_ENABLE) will be made at the
>>> start of the process. The mode is included in the flags for this syscall.
>>>
>>> The Scope determines which parts of the tagging implementation in Glibc
>>> are actually activated.
>>>
>>> This may appear complicated however this is due to the difference of how
>>> heap tagging is implemented compared to tagging of stack and globals. In
>>> the latter case binary needs to be built with necessary instrumentation
>>> that is not controlled by Glibc.
>>>
>>> Any feedback welcome!
>> [1] https://sourceware.org/git/?p=glibc.git;a=shortlog;h=refs/heads/azanella/aarch64-memtag
> 
> Hope this clarifies it!
> 
> Thanks,
> Yury
>