Re: Question regarding IPIQs

This is the right list.

Max


On Wed, Aug 28, 2013 at 10:49 AM, Christiano F. Haesbaert <
[email protected]> wrote:

> Ping, if I used the wrong list please point me to the correct one.
>
>
>
>
> On 22 August 2013 17:22, Christiano F. Haesbaert <[email protected]>wrote:
>
>> Hi there,
>>
>> I've been studying dragonfly for a while now and this is something I
>> could never figure out, I'm pretty sure I'm missing something and I'd
>> appreciate if someone could give me some pointers.
>>
>> Suppose the following cenario.
>>
>> Cenario A [A]
>> cpu0: on whichever context
>> cpu0: spin_lock(&a) (now in a critical section)
>> cpu0: wakeup(ident1) (ident1 has a sleeper on cpu1)
>> cpu0: lwkt_send_ipiq3(cpu1, wakeup(ident1))
>>
>> But ipiq for cpu1 is full, so I enable interrupts, although I'm still on
>> a critical section, process my own ipiq and spin until cpu1 queue is not
>> full, but at this point I'm still holding spin_lock a.
>>
>> This is the code I'm referring to, lwkt_ipiq:221-229
>>
>>         while (ip->ip_windex - ip->ip_rindex > MAXCPUFIFO / 4) {
>>             if (atomic_poll_acquire_int(&target->gd_npoll)) {
>>                 logipiq(cpu_send, func, arg1, arg2, gd, target);
>>                 cpu_send_ipiq(target->gd_cpuid);
>>             }
>>             KKASSERT(ip->ip_windex - ip->ip_rindex != MAXCPUFIFO - 1);
>>             lwkt_process_ipiq();
>>             cpu_pause();
>>         }
>>
>> cpu1: ipiq is full and is running an ithread, so it's in a critical
>> section.
>> cpu1: spin_lock(&a);
>> cpu1: ??deadlocked??
>>
>> cpu1 will never process its own ipiq, therefore cpu0 will never make
>> progress,
>> since cpu0 holds spin_lock a, cpu1 will never make progress as well.
>>
>> Would this imply that a code that may generate an ipi down the stack, may
>> never
>> hold a spinlock ? I understand it is very likely that there is no current
>> path
>> that does a wakeup holding a spinlock.
>>
>> Furthermore, in this cenario you end up processing your own ipiq even if
>> you
>> were already on a critical section, can't the ipiq callbacks actually race
>> against the code that lead to
>> wakeup()->lwkt_sent_ipiq3()->lwkt_process_ipiq() ?
>>
>> Cenario [B]
>> Still regarding the paragraph above, what if both cpu0 and cpu1 have
>> their ipiqs
>> full _and_ come from a code in a critical section, as in:
>>
>> cpu0: cpu0 own ipiq is full
>> cpu0: crit_enter()
>> cpu0: crit_enter()
>> cpu0: wakeup(ident1) (ident1 has a sleeper on cpu1)
>> ...
>>
>> cpu1: cpu1 own ipiq is full
>> cpu1: crit_enter()
>> cpu1: crit_enter()
>> cpu1: wakeup(ident0) (ident0 has a sleeper on cpu0)
>> ...
>>
>> Both realize the other is full and process their own queues, but they
>> break the
>> atomicity required by the two calls to crit_enter() earlier
>>
>> What am I missing ? (Sorry for the noise).
>>
>> Thanks
>>
>> --
>> Christiano Farina HAESBAERT
>> Do NOT send me html mail.
>>
>
>