Re: Patches to allow usefdt mode that works on a 2 socket PowerMac3, 6 example too --and makes more work on 2-socket/1-core-each PowerMac11, 2

[I have added an investigatory patch that so far has stopped
the stuck-sleeping problem.]

On 2019-Apr-15, at 00:06, Mark Millard <marklmi at yahoo.com> wrote:


> On 2019-Apr-13, at 11:39, Mark Millard <marklmi at yahoo.com> wrote:
> 
>> [My adjustment to fdt_add_subnode_namelen was inept.]
>> 
>> On 2019-Apr-12, at 16:17, Mark Millard <marklmi at yahoo.com> wrote:
>> 
>>> On 2019-Apr-12, at 14:20, Dennis Clarke <dclarke a t blastwave.org> wrote:
>>> 
>>>> On 4/12/19 4:51 PM, Mark Millard wrote:
>>>>> On 2019-Apr-12, at 13:13, Dennis Clarke <dclarke at blastwave.org> wrote:
>>>>>> On 4/12/19 3:19 PM, Mark Millard via freebsd-ppc wrote:
>>>> .
>>>> .
>>>> .
>>>>>> 
>>>>>> Would you be so kind as to paste all this into :
>>>>>> 
>>>>>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=233863
>>>>>> 
>>>>>> Really I would like to run some tests and follow up in the bug reports.
>>>>> Okay I'll paste them in as attachments. But be warned:
>>>> 
>>>> Fair warning received loud and clear :-)
>>>> 
>>>>> The 2 files do not deal with threads being stuck sleeping
>>>>> (and, so, the fans going) or other such. The stuck-sleeping
>>>>> problem happens for both multi-socket G5's and multi-socket
>>>>> G4's. (I do not have access to single-socket multi-core
>>>>> powerpc64 or powerpc machines to test.)
>>>> 
>>>> I have multiple G5 type boxen and will try them out. At least try
>>>> to.
>>>> 
>>>>> So do not expect too much from these patches: They address
>>>>> some necessary issues but are not sufficient for everything.
>>>>> 
>>>> 
>>>> Of course. No problem.
>>>> 
>>>> 
>>>>> These patches for the openfirmware->fdt translation are
>>>>> closer to being reasonable for FreeBSD official use
>>>>> than my highly context-specific stuck-sleeping patches for
>>>>> usefdt mode.
>>>> 
>>>> Well to be frank we know this is for mac g5 hardware and thus having
>>>> them working at all in any fashion is better than the current situation.
>>>> Apple made a ton of them and they are dirt cheap and available as
>>>> opposed to the IBM Power situation which is expensive and just in
>>>> datacenters.
>>> 
>>> 
>>> I have added another attachment with patches for having hang-ups
>>> at AP startup happen less often. These are in AIM-specific code
>>> and so has less of a chance of causing other contexts problems.
>>> They are also powerpc64 specific. Again, the patches are
>>> investigatory and not in a form for direct check-in to FreeBSD.
>>> 
>>> This pair of patches narrows the time period over which threads
>>> from the stages:
>>> 
>>>      SI_SUB_KTHREAD_INIT     = 0xe000000,    /* init process*/
>>>      SI_SUB_KTHREAD_PAGE     = 0xe400000,    /* pageout daemon*/
>>>      SI_SUB_KTHREAD_VM       = 0xe800000,    /* vm daemon*/
>>>      SI_SUB_KTHREAD_BUF      = 0xea00000,    /* buffer daemon*/
>>>      SI_SUB_KTHREAD_UPDATE   = 0xec00000,    /* update daemon*/
>>>      SI_SUB_KTHREAD_IDLE     = 0xee00000,    /* idle procs*/
>>> #ifndef EARLY_AP_STARTUP
>>>      SI_SUB_SMP              = 0xf000000,    /* start the APs*/
>>> #endif 
>>> 
>>> can conflict with starting an AP via an slb replacement position
>>> picked via expressions like mftb()%n_slbs . It does this by 
>>> explicitly picking and setting up a slb slot for its use just
>>> before starting the AP.
>>> 
>>> (The AP has to be part way along before it can do its own
>>> automatic-random-slb-slot-replacements from what I can tell.)
>>> 
>>> The patches do not remove the race and still do sometimes fail to
>>> prevent getting a hang-up on a AP start. But it greatly decreased
>>> the rate of hangups in my testing. (So it is a good source of
>>> evidence about the original problem.)
>>> 
>>> If EARLY_AP_STARTUP was supported and used, the AP startup would
>>> not have hang-up problems from mftb()%n_slbs based slb
>>> replacements for other threads.
>>> 
>>> The patches are a hack, rather than a general/complete fix --and
>>> I do not expect to see them in FreeBSD. But they do help set up
>>> a better context for investigating other things.
>> 
>> The disabling of blocking duplicate paths in fdt_add_subnode_namelen
>> was done incorrectly. I'll replace the attachment after building
>> and testing. I think this is the explanation for the PowerMac11,2
>> shutdown -r or -p problems.
>> 
>> The code should have just disabled the return, more like:
>> 
>>       if (offset >= 0)
>> #if 0
>> // Some Macintoshes have identical package-to-pathname results for
>> // multiple nodes of the same type and unit under the parent node.
>> // Avoid blocking this for fdt.
>>               return -FDT_ERR_EXISTS;
>> #else
>>               ;
>> #endif
>>       else if (offset != -FDT_ERR_NOTFOUND)
>>               return offset;
>> 
>> Instead the messed up change did the "return offset;" and
>> so did not do the addition of the node, instead returning
>> the pre-existing one to be manipulated.
> 
> 
> I've managed to boot the 2-socket/1-core-each G5 PowerMac7,2 in
> usefdt mode. I've added another attachment for patching 3 more
> files, also shown below:
> 
> Index: /usr/src/sys/powerpc/powermac/hrowpic.c
> ===================================================================
> --- /usr/src/sys/powerpc/powermac/hrowpic.c     (revision 345758)
> +++ /usr/src/sys/powerpc/powermac/hrowpic.c     (working copy)
> @@ -169,7 +169,7 @@
>        hrowpic_write_reg(sc, HPIC_ENABLE, HPIC_SECONDARY, 0);
>        hrowpic_write_reg(sc, HPIC_CLEAR,  HPIC_SECONDARY, 0xffffffff);
> 
> -       powerpc_register_pic(dev, ofw_bus_get_node(dev), 64, 0, FALSE);
> +       powerpc_register_pic(dev, OF_xref_from_node(ofw_bus_get_node(dev)), 64, 0, FALSE);
>        return (0);
> }
> 
> Index: /usr/src/sys/powerpc/powermac/uninorth.c
> ===================================================================
> --- /usr/src/sys/powerpc/powermac/uninorth.c    (revision 345758)
> +++ /usr/src/sys/powerpc/powermac/uninorth.c    (working copy)
> @@ -181,7 +181,7 @@
>            <= 0)
>                panic("Interrupt but no interrupt parent!\n");
> 
> -       if (OF_searchprop(iparent, "#interrupt-cells", &icells, sizeof(icells))
> +       if (OF_searchprop(OF_node_from_xref(iparent), "#interrupt-cells", &icells, sizeof(icells))
>            <= 0)
>                icells = 1;
> 
> Index: /usr/src/sys/powerpc/powerpc/openpic.c
> ===================================================================
> --- /usr/src/sys/powerpc/powerpc/openpic.c      (revision 345758)
> +++ /usr/src/sys/powerpc/powerpc/openpic.c      (working copy)
> @@ -37,6 +37,8 @@
> #include <sys/sched.h>
> #include <sys/smp.h>
> 
> +#include <dev/ofw/openfirm.h>
> +
> #include <machine/bus.h>
> #include <machine/intr_machdep.h>
> #include <machine/md_var.h>
> @@ -220,7 +222,7 @@
>        for (cpu = 0; cpu < sc->sc_ncpu; cpu++)
>                openpic_write(sc, OPENPIC_PCPU_TPR(cpu), 0);
> 
> -       powerpc_register_pic(dev, node, sc->sc_nirq, 4, FALSE);
> +       powerpc_register_pic(dev, OF_xref_from_node(node), sc->sc_nirq, 4, FALSE);
> 
>        /* If this is not a cascaded PIC, it must be the root PIC */
>        if (sc->sc_intr == NULL)


The patch is:

# svnlite diff  /usr/src/sys/powerpc/powerpc/mp_machdep.c | more
Index: /usr/src/sys/powerpc/powerpc/mp_machdep.c
===================================================================
--- /usr/src/sys/powerpc/powerpc/mp_machdep.c   (revision 345758)
+++ /usr/src/sys/powerpc/powerpc/mp_machdep.c   (working copy)
@@ -77,9 +77,10 @@
        PCPU_SET(awake, 1);
        __asm __volatile("msync; isync");
 
+       powerpc_sync();
        while (ap_letgo == 0)
-               __asm __volatile("or 31,31,31");
-       __asm __volatile("or 6,6,6");
+               powerpc_sync();
+       isync();
 
        /*
         * Set timebase as soon as possible to meet an implicit rendezvous
@@ -262,8 +263,11 @@
        __asm __volatile("msync; isync");
        
        /* Let APs continue */
-       atomic_store_rel_int(&ap_letgo, 1);
+       ap_letgo= 1;    // depend on prior sync, no need to lwsync first
 
+       powerpc_sync(); // analogous to what the ap's do (more similar time frame?)
+       if (ap_letgo) isync();
+
        platform_smp_timebase_sync(ap_timebase, 0);
 
        while (ap_awake < smp_cpus)



===
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)

_______________________________________________
[email protected] mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ppc
To unsubscribe, send any mail to "[email protected]"