Re: Battles with CountedCompleter

Sorry I sent the email with the wrongly subscribed address :-(
See my answer below.

I hope it helps!

Romain Colle
atoti R&D Director at ActiveViam

On Tue, Sep 29, 2020 at 11:03 AM Romain Colle <[email protected]> wrote:

> Hi Shevek,
>
> It might help to perform the recursive split as a fork() within the parent
> task code instead of using external submits.
>
> Looking at ForkJoinPool.awaitJoin(), it seems that a waiting task only
> tries to steal from queues at odd indexes (worker internal queues), which
> makes sense since they are the only queues where they should be able to
> find subtasks.
> Therefore a running task will never try to steal from a submission queue,
> and the only way to achieve parallelism is for them to compensate and park
> themselves.
>
> For more details see:
>
> https://github.com/openjdk/jdk/blob/6bddeb709d1d263d0d753909cabce7e755e7e27d/src/java.base/share/classes/java/util/concurrent/ForkJoinPool.java#L356-L360
>
> https://github.com/openjdk/jdk/blob/6bddeb709d1d263d0d753909cabce7e755e7e27d/src/java.base/share/classes/java/util/concurrent/ForkJoinPool.java#L1699
>
> Let me know if this helps.
> Thanks,
> Romain
>
>
> On Fri, Sep 25, 2020 at 2:04 AM Shevek via Concurrency-interest <
> [email protected]> wrote:
>
>> I have been doing battle with CountedCompleter, and I'm stuck at a point
>> where I'm doing something like this:
>>
>> doInParallel(Iterable<thing> tasks) {
>>    CountedCompleter joinTask = new CountedCompleter();
>>    for (some unknown number of things)
>>        pool.submit(new CountedCompleter(parent, ...));
>>    joinTask.tryComplete();
>>    joinTask.join();
>> }
>>
>> The objective is to have a recursively-safe construct like:
>> try (ForkJoinScope scope = new ForkJoinScope(pool, ...)) {
>>      for (whatever)
>>          scope.execute(task);
>> } // close() calls join()
>>
>> Any subtask may itself repeat this pattern. The trouble I'm having is
>> that sometimes I get a lot of threads blocked here:
>>
>> "ForkJoinPool-1-worker-6" #19 daemon prio=5 os_prio=0
>> tid=0x00007f39e5075800 nid=0xd0c in Object.wait() [0x00007f39515f7000]
>>   java.lang.Thread.State: WAITING (on object monitor)
>> at java.lang.Object.wait(Native Method)
>> at java.util.concurrent.ForkJoinTask.internalWait(ForkJoinTask.java:311)
>>          - locked <0x0000000477c78528> (a
>> org.compilerworks.common.util.concurrent.ForkJoinScope$JoinTask)
>> at java.util.concurrent.ForkJoinPool.awaitJoin(ForkJoinPool.java:2058)
>>        at
>> java.util.concurrent.ForkJoinTask.doJoin(ForkJoinTask.java:390)
>> at java.util.concurrent.ForkJoinTask.join(ForkJoinTask.java:719)
>>
>> What I can't work out is why these blocked threads aren't helping? 6 of
>> the threads in my 12-thread pool are blocked, and 6 are working.
>> Eventually, every so often, one of them seems to unblock. I'm trying to
>> trace the logic in the code to work out why the blocked threads don't
>> simply steal other work and do it. I've tried unit testing my
>> wrapper/controller code twenty ways up and it doesn't block in tests,
>> but it fails in application.
>>
>> Inspection of the heap of a blocked task shows (for example)
>>
>> * joinTask.pending=32
>> * 33 ForkJoinTask instances have a pointer to this joinTask as the
>> completer.
>> * joinTask and its children are in the correct ForkJoinPool
>> * Java 1.8.0_252
>>
>> Can anybody please help? I'm happy to submit exact code as there are a
>> couple of nuances to what I'm doing which might be relevant.
>>
>> I'm willing to be polite, but not effusive about the documentation of
>> CountedCompleter and similar, so it's entirely possible that I'm using
>> an API wrong.
>>
>> Code is attached.
>>
>> Thank you.
>>
>> S.
>> _______________________________________________
>> Concurrency-interest mailing list
>> [email protected]
>> http://cs.oswego.edu/mailman/listinfo/concurrency-interest
>>
>

_______________________________________________
Concurrency-interest mailing list
[email protected]
http://cs.oswego.edu/mailman/listinfo/concurrency-interest