Re: AssertionError on barrier addWaiter

Gary Mulder <[email protected]> wrote:

> On 5 May 2017 at 08:02, Amaury Denoyelle <[email protected]> wrote:
> >
> > Do you have any idea of the cause of the problem ? Is 10,000 threads
> > waiting on one barrier a little bit overkill ?
> >
> I can't speak about your barrier exception, but I can say that 5000 threads
> per server is quite a large number for the JVM to manage. With reasonably
> simple test code I usually have a rule of thumb of no more than 100 threads
> per core with random sleeps between HTTP requests. This would imply a 50
> core server for 5000 concurrent threads.
>
> However you may have a bigger problem if a many of those 5000 threads are
> being synchronised at a barrier. You say that the threads aren't
> synchronised, but with that number of threads the Linux scheduler is likely
> being hammered as many threads become ready to run at the same time. A 16
> core hyper-threaded server has 32 hyper-cores, so the Linux scheduler can
> physically only concurrently schedule 32 threads. If you're using Windows
> the problem is likely worse as the Linux scheduler is a lot more
> lightweight than the Windows scheduler.
>
> Finally, keep an eye on your JVM Garbage Collection. With some many threads
> creating objects your GC is likely running quite hot.
>
> In all, you will be fighting a lot with your test execution performance and
> need to prove to yourself that it isn't the primary bottleneck, otherwise
> your test time will reflect the performance of your test harness and not
> system under test.

Thank you for your quick answer.

I currently use two agents, with 16 cores on each. You are maybe right
on the fact that it is too much for the system, but I really need 10,000
threads. I will maybe try to add more agents, but I do not have
countless of these servers.

That being said, I find that the tests on the agents are accurate. I do
not measure precise timer, but just check some basic conditions and all
10,000 tests are finely executed. All agents are doing their works fine.
The problem I encounter is on the console, on a dedicated host without
other processes so that confuses me a little.

-- 
Amaury Denoyelle

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot