Re: GETF Speedup?

OK, I have a bit more time now:

BLUF (bottom line up front): my preferences are set by my research interests, and of course, may not align with yours.

Sorry if this is a bit scattershot. I have *some* time, but not enough to turn this into a full coherent essay.

> On Oct 16, 2025, at 6:04 AM, Yuri Davidovsky <[email protected]> wrote:
> 
> 
> 
>> On 15 Oct 2025, at 20:27, Bradford Miller (as bradford dot w dot miller at gmail dot com) <[email protected]> wrote:
>> 
>> Yes, RISC v CISC. And they pushed caching. But that doesn’t help with GUPS (if anything, caching hurts). If all you are interested in is making your existing imperative codebase faster, then yes, you are probably better off investing in “sufficiently smart compilers”. If you are instead interested in alternative architecture and algorithms to avoid von Neumann bottlenecks, compilers won’t help as much as more control over the hardware.
> 
> I have a lot of questions about this (I am kinda with Bradshaw on this one for once) as I have little reason to doubt that modern machines are substantially better than what we had, say, 40 years ago in every respect. So the questions are:
> 
> 1. What is GUPS?

As Tim reported, “Giga Updates Per Second”, a common benchmark that measures how quickly a CPU can service memory (read-modify-write) when caches won’t help (because it’s completely random word-at-a-time access). This is mostly of interest to the HPC crowd, as CPUs have outstripped memory speed improvements, thus most edge of the art CPU cores are in a constant state of memory starvation.

> 2. In which ways caching can hurt (especially given the current speed gap between CPU and RAM in modern machines)?

Caches can help only if you have sufficient locality of reference, e.g., if you need something on a page, you’ll also need other things from that page. There are a large class of problems, typically assigned to supercomputers, that break that assumption. I’m not trying to build a supercomputer (well maybe in a historic sense), but I’ll talk about the kind of problems I’m interested in further down.

> 3. Why does Common Lisp code benefit less from smart compilers than imperative languages (I assume C is meant here as an imperative language)?

So certainly one can use CL as an imperative language, but anyway I probably misstated this. What I should have said that dynamic languages would benefit less from smart compilation, but they would still get some benefit. I’ll explain this further when I address my own research, but the main idea is that most approaches to application development, today, segregates “offline” and “online” work. By pushing more effort into “offline” work (such as, typically, compilation), we can save time and/or space while working “online”. This is the sort of paradigm where smart compiling helps, because the smarter you make the compiler, the more resources it needs so you want that to happen offline. If, like me, you need to do meta programming (that is write programs that manipulate and
  rewrite other programs online, for which the Lisp family of languages is eminently suitable) that paradigm breaks down.

> 4. What are the von Neumann bottlenecks that need to be avoided and why alternative architectures are need for that?

The von Neumann bottleneck refers to the mismatch between CPU speed and memory speed forcing caching or wait states to be inserted (in pipelined architectures, one typically gets bubbles or voids in the pipeline). Alternative architectures can try to address this, either by topology (e.g. hypercubes), partitioned global address spaces, having a DMA type sub processor that can do scatter/gather into a particular CPU’s memory,  and others. It’s an area of active research, and the best solution usually depends on the particular application problem. 

> 5. In which way control over hardware is superior to control over compilers? (I am not sure the phrasing is connect as it feels similar to saying “in what ways water melons are superior to vacuum cleaners” but that is what the quoted text kinda says, it seems.)

So the short story here is a very active research area particularly in Europe: dynamically reconfigurable computing. Most modern FPGAs allow you to dynamically reconfigure them while they are active, by designating an area on the FPGA fabric for reconfiguration and then loading a bit file which will define the logic for that area. Of course there is some cost to doing that (if you don’t have a very small FPGA then loading a bit file can be a relatively slow process and itself is an example of a von Neumann bottleneck because it’s a serial transfer).  See https://en.wikipedia.org/wiki/Reconfigurable_computing

One way to think about this is setting up your processor like a paged memory system, except you can swap in processing elements (like ALUs or communication support) rather than just data (or programs). Thus your efficiency can be tuned to the problem you are working on, rather than being stuck with a jack-of-all-trades general purpose CPU. 

So given a particular budget (of board space or chip layout or whatever), rather than being committed to a particular set of engineering tradeoffs, you can change them. For example, if you have a bunch of FPGA fabric dedicated to cache but are hit with a GUPS-style distributed update problem, you can get rid of the cache and instead add other elements that would actually be useful.

Of course there are a number of problems once you decide you want to go down this kind of path, such as not just knowing your current workload, but having to predict your future workload so you can, if desired, schedule a reconfiguration.

> I feel like we are lacking a huge amount of backstory behind the terse email so I want to unpack it a little.

TL;DR - if you got this far, I’m already somewhat surprised, but the rest really just talks about my own research interests, and not really a common-lisp specific kind of thing, which is why I tend to be terse.

Anyway, the basic family of problems I’m interested in investigating is long-lived agents (e.g., a robot) that gets put into novel situations and has to not only plan to solve problems, but has to discover what the problems are in the first place.

Examples: autonomous space exploration, warfare, plumbing, debugging, pretty much any non-trivial human activity that involves reason, *timely* action and a partially observable environment.

Now since we don’t know what we’re going to do a priori, we start out basically with some knowledge about ourselves and a little bit of physics in the environment. 

In the JDL sensor fusion model (see, e.g. https://www.sensortips.com/featured/sensor-fusion-levels-and-architectures-faq/) we generally don’t have to worry about Level 0 fusion, and we may have some priors we can leverage at L1 and L2, but above that we are on our own, and we generally need to get to an effective L3 for the environment. 

As we explore the environment and our own capabilities in that environment, we have to manage our resources: power, latency (timeliness for reacting to change or perceived threats), etc. 

The way I’m currently looking at doing that is leveraging dynamic reconfiguration with a PIM (processor in memory) architecture. The PIM is one way to avoid the von Neumann bottleneck, because as you add memory, you also add processors. Think of a Turing Machine that doesn’t just have an infinite memory tape, but an infinite processor tape each of which have finite memories :-). 

Reconfiguration lets us analyze a problem, then reconfigure ourselves to solve it efficaciously. We need that because the environment is continually changing, so latency is a big issue. We also need to segregate our processing (effectively take advantage of concurrency) so our attentional model can run independently of anything else we’re working on (e.g., notice a pipe is about to fall on our “head"). Some of this is based on so-called “black box theorem provers”, which essentially solved satisfiability problems (probabilistically), by first reducing them to an FPGA configuration, then running the FPGA. This turned out to be much more effective than software-only solutions (at least at the time, I’m not sure what the tradeoffs are today, but the algorithms should still work). 

But once we’ve reconfigured, we have another problem - we may need to regenerate our processes for the new configuration (we may want to do that anyway thanks to the 80/20 rule and our need for timeliness) and that means recompilation. So compilation becomes an online rather than an offline task. To some sense we can offload that to microcode that is part of the reconfiguration machine (that is the program that generates new configurations).

We need to be able to reprioritize our tasks, and, essentially, be able to quickly generate partial orderings of desiderata based on multiple criteria, which may not be known a priori, and whose weights will shift depending on both environmental and internal factors (e.g., running low on power, environment rate of change, external communication, etc.). This is like being asked to sort a large number of photographs based on one or more criteria, like “most to least similar background to this picture” or “evokes most to least similar emotions to this picture” or “depicts paradigm change” (I just threw that one in to make it harder ;-). The key is you don’t know the criteria a priori.

But that gives us a long list of problems, not all of which (or even many of them) I can hope to solve. Luckily, some are being worked on by others. 

Right now I’m focused on the PIM architecture, how to program it, etc. And running the PIMs like a form of smart content accessible memory (you get a evaluation function, return the n highest scores across all the memories).

If I took a RISC based approach to building the PIM, note that the RISC processor doesn’t decode its instructions: they essentially ARE the microcode. But the problem is that a given (RISC) compiler backend doesn’t support reconfiguration: the processor changed, so *that* code won’t do what we expect. We need that indirection (instruction decode) to deal with mapping machine instructions to running the chip, or we need to generate a new backend after every reconfiguration and then recompile everything. (You thought GC was a problem?). It also limits the ability for “smart compilation” to help, because the smart compiler needs to know the processor architecture. But that can change at any time! Maybe at point t1 there are 64 128 bit registers, but at point t2 there are 32 64 bit r
 egisters. Maybe the floating point accelerator went away. Maybe registers used to have two busses, but now have 5 with three more ALUs. Etc.

Static solutions do not lend themselves to dynamic situations. 

The “big experiment” I’m working toward is a variation on the inverse pendulum problem. Except there are multiple pendulums, they are partially tied together with springs, you have sensors which can see different parts of the room (blind men and the elephant), and finally some grad student walking around and disturbing it in various annoying ways. 

It’ll be a while before I get there.

Back to LIspM’s: Genera, in its later iterations have a very useful deadline scheduler too. And stack groups were useful for capturing dynamic state, not just threading. (Admittedly, InterLisp was better in that the stack was a 1st class object, we did have DLions running InterLisp too). 

Back to GETF: all software, but the Symbolics approach I previously referenced is related to the "self-reconfiguration depending on your data", problem. 


_______________________________________________
Lisp Hug - the mailing list for LispWorks users
[email protected]
http://www.lispworks.com/support/lisp-hug.html