Re: Anyone working on Octeon SMP?

On 10/04/2026 11:18, Kevin Bowling wrote:
> On Mon, Apr 6, 2026 at 6:42=E2=80=AFAM Nick Hudson =
<[email protected]> wrote:
>>=20
>> On 06/04/2026 10:49, Kevin Bowling wrote:
>>> On Sat, Mar 28, 2026 at 8:28=E2=80=AFAM Andrew Parker =
<[email protected]> wrote:
>>>>=20
>>>> On Thursday, 19 March 2026 23:31:36 EDT Kevin Bowling wrote:
>>>>> On Sun, Feb 2, 2025 at 6:52=E2=80=AFAM Andrew Parker =
<[email protected]> wrote:
>>>>>> On 1/29/25 06:23, Nick Hudson wrote:
>>>>>>> On 20/12/2024 22:01, Andrew Parker wrote:
>>>>>>>> Hi, I've been working towards getting a MULTIPROCESSOR build =
more
>>>>>>>> stable
>>>>>>>> on my ER4 and would like to know if anyone else may be working =
on the
>>>>>>>> same?
>>>>>>>>=20
>>>>>>>> I've found a few areas that were causing instability on my =
machine and
>>>>>>>> have some patches that help (but mostly just for debugging at =
this
>>>>>>>> point). If there's any interest in teaming up and exchanging =
ideas or
>>>>>>>> patches for SMP on Octeon please let me know.
>>>>>>>=20
>>>>>>> Sure.
>>>>>>>=20
>>>>>>> I've dropped the ball on this and said I had a couple of fixes =
in mind,
>>>>>>> but done nothing to share them. Hopefully we can make it stable.
>>>>>>>=20
>>>>>>> Nick
>>>>>>=20
>>>>>> Great!  I'm curious about what you have in mind for improvement.  =
I've
>>>>>> mostly been looking around TLB invalidation and perhaps a missing =
memory
>>>>>> barrier.
>>>>>>=20
>>>>>> Anyway, I'll work on getting some stuff cleaned up and contact =
directly
>>>>>> if that works for you.
>>>>>=20
>>>>> I'm interested in this as well, is there anything to share around
>>>>> current status or issues as well as if anything is pending out of
>>>>> tree?
>>>>=20
>>>> I hoped to spend more time on this over the winter but ended up =
moving and
>>>> some of my networking gear is still packed up.
>>>>=20
>>>> It's been a slow process getting my test environment setup again =
but it would
>>>> be great to pick this back up...especially if there's continued =
interest in
>>>> it.
>>>>=20
>>>> Give me a week or two to see what I can dig up and I'll be in =
touch.
>>> I've made some progress but it turned into a much deeper hole than I
>>> was anticipating.  I can increase SMP stability with some changes in
>>> pmap_tlb.c to add some icache syncs
>>=20
>> I have a change I'll commit soon to handle EXECness (more) correctly.

This is committed

https://mail-index.netbsd.org/source-changes/2026/04/10/msg161522.html


>>=20
>> Am I right in thinking at least some octeon processors icaches are =
VIVT?
>> I've forgotten most of the mips stuff I knew... If so there will be =
more flushing
>> required for VIVT.
> VIPT.  But it has an assortment of fun issues.

fun issues?



>>> and guarding around an assert that
>>> is easy to trigger
>>> @@ -735,7 +767,9 @@ pmap_tlb_shootdown_bystanders(pmap_t pm)
>>>                         * And best of all, we avoid an IPI.
>>>                         */
>>>                        KASSERT(!kernel_p);
>>> -                       pmap_tlb_pai_reset(ti, pai, pm);
>>> +                       if (pai->pai_asid > KERNEL_PID) {
>>> +                               pmap_tlb_pai_reset(ti, pai, pm);
>>> +                       }
>>=20
>>=20
>> you mean this KASSERT in pmap_tlb_pai_reset?
> Yeah.  But let me think harder about this.
>>    252  /*
>>    253   * We must have an ASID but it must not be onproc (on a =
processor).
>>    254   */
>>    255  KASSERT(pai->pai_asid > KERNEL_PID);
>>=20
>>> I think there are a variety of pmap changes needed.  But I wonder if
>>> any MIPS SMP or other PMAP_TLB_NEED_SHOOTDOWN has been heavily
>>> exercised?
>>=20
>> Almost certainly not.
> Good to know, I was a bit shy about looking at the MI pmap at first.
>>> IPI and TLB stuff is probably "simpler" on ARM.  On
>>> FreeBSD (13), OpenBSD, Linux the MIPS TLB shootdowns are =
synchronous.
>>=20
>> No other NetBSD architecture (not sure of powerpc booke status =
actually) uses
>> the PMAP_TLB_NEED_SHOOTDOWN stuff.
>>=20
>> I'm about to switch aarch64 to sys/uvm/pmap which uses architecture =
defined
>> broadcast TLB operations. RISC-V uses SBI remote fence operations.
> Arch defined broadcast sounds very suitable for what we'll need to do
> here.  Is that out of tree?

Arm architecture defined the broadcast TLB operations long ago. Others =
have caught up...

MI PMAP uses tlb_* functions
=
https://nxr.netbsd.org/xref/src/sys/arch/aarch64/aarch64/aarch64_tlb.c#84
=
https://nxr.netbsd.org/xref/src/sys/arch/aarch64/aarch64/cpufunc_asm_armv8=
.S#221