Re: Performance assistance / advice

Rather than just tackling the obvious “every particle affects every other particle”, I’d want to examine the underlying physics to see if the force law diminishes rapidly enough to where I could get away with restricting examination of particle interactions to some degree of nearest neighbors.

If not, then you likely have a force field that is non-integrable, and possibly un-physical.

But if the force field does decay rapidly enough with distance, then we can do a cheaper algorithm. Sure it won’t be 100% fidelity, but maybe good enough, and still be tractable. Perhaps also incorporate propagation delays, since particles further out can’t possibly affect us immediately with their changes.

IOW - if you can’t first work out the answer on the back of an envelope to first order, then no point spending more time and money on the problem.



> On Mar 12, 2025, at 04:20, Tim Bradshaw (as tfb at cley dot com) <[email protected]> wrote:
> 
> I really don't want to get into a C supremacy argument.  C is like heroin: it has its uses, but I've seen it do so much harm to so many people that I'd rather not go there if I can possibly avoid it.  The kind of microbenchmarky things C is so good at are just never the problem for big physics models: if you want to make your million-line model run faster the problem is always how to scale it, and never how to fix some tiny inner loop.  Scaling is also a genuinely interesting problem.
> 
> I'm very much with David McClain here, and I really would like not to get into a big discussion about C vs Lisp.
> 
> I don't really have the patience to write a C program which is the equivalent of what my code macroexpands into.  I am wondering about doing it in Fortran: there's an LLVM-backed Fortran compiler which should be quick and Fortran has language-level parallelism now, although a lot of it is oriented towards message-passing shared-nothing systems of course, and my algorithm wants to be SMP: message-passing on top of an SMP system is likely to suck because it will copy a lot of arrays which don't need to be copied.
> 
> However, if anyone else wants to, here's the challenge: simulate a system of gravitating point particles using the obvious horrid quadratic algorithm: every particle sums the acceleration due to every other particle and then updates its position and velocity based on the resulting acceleration for a given delta-t.  You're not allowed to use a cleverer integrator!
> 
> It needs to be correct: a good way of checking that it probably is doing so is to measure the initial and final energy to check it is conserved.
> 
> Rather than worrying about FLOPS, a more physically interesting measure of performance is the rate: how many seconds per second it runs at.  For a system of 5,000 particles, with a delta-t of 1 second, on an M1, my program has a rate of about 54 to 58, so it's simulating that system in between 54 to 58 times real time.  The rate falls below 1 somewhere around 35,000 particles.  So that's what anything needs to beat.
> 
> (The reason for the 'between 54 and 58' is that it looks like the machine actually starts slightly throttling pretty quickly: if I run it for 1,000 steps (so about 17 seconds real time) I get 58, but 10,000 steps (3 minutes or so) is significantly lower, and you can see the rate fall over time.  I read the thing about disabling throttling, but there's usually a good reason for keeping machines inside their designed thermal envelope: I'll keep the battle shorts in reserve for now.)
> 
> --tim
> 
>> On 11 Mar 2025, at 20:42, Yuri Davidovsky (as work at disclosure dot ie) <[email protected]> wrote:
>> 
>> 
>> 
>>> On 11 Mar 2025, at 21:31, Tim Bradshaw (as tfb at cley dot com) <[email protected]> wrote:
>>> 
>>> Well, this is actually my claim here: yes, like C fast.  I am secretly hoping that some masochist will write a version of my algorithm in C, so I can verify this.
>> 
>> Unfortunately, it won’t be the case in all situations. For example, as outlined above, gcc is quite good at automatic vectorisation, Lisps’ capabilities at that are rather rudimental at this point in time, as I understand. Additionally, when coding in C you can have access to intrinsics (wrapper functions for underlying CPU instructions) which can give you a bit of an edge when optimising for a specific platform, for example the rbit instruction on arm64 that does bit reversal and that is strangely not present on x64 and which can save a lot of cycles.
>> 
>> Moreover, you do not even explicitly need to use the intrinsics, gcc can recognise the bit reversal algorithm patterns and employ the rbit instruction even without you knowing about its existence. At the moment such things are not possible in lisps, so in some cases inability to match C’s performance is going to be difficult to even explain, let alone to replicate.
>> 
>> _______________________________________________
>> Lisp Hug - the mailing list for LispWorks users
>> [email protected]
>> http://www.lispworks.com/support/lisp-hug.html
> 
> 
> _______________________________________________
> Lisp Hug - the mailing list for LispWorks users
> [email protected]
> http://www.lispworks.com/support/lisp-hug.html


_______________________________________________
Lisp Hug - the mailing list for LispWorks users
[email protected]
http://www.lispworks.com/support/lisp-hug.html