Re: Disgusting hacks [was Re: I wonder if I'm missing something...]
"David McClain (as dbm at refined-audiometrics dot com)" <[email protected]>
| Newsgroups | gmane.lisp.lispworks.general |
|---|---|
| Message-ID | <[email protected]> |
Many years ago on Apple OSX there was a facility for inter-process data transfer that was used by Aquaterm to display graphics output from any other program. I used that system for a while from my Lisp, before I wrote my current Plotter package. The IPC was handled by Objective C code. Looking around, it looks like the system still exists and is called Apple XPC? I think it was intended solely for IPC between processes on the same host machine. Underneath XPC they must be using Berkeley Sockets (or whatever they have morphed into by now). And so the only other layer provided by Apple must have been data serialization and deserialization. Compared to Lisp data types, that would represent a pretty arid environment to have to live within. We have a very good system for Lisp data serialization and deserialization. Written years ago by Sean Ross, and I have since added numerous facility extensions of my own. I forget what he called it… Maybe CL-STORE?? By using your socket I/O we can extend the XPC facility to cross-machine transfers. I already do that in my Actors system using Asynchronous TCP/IP as the communication layer. Your version looks like it is Synchronous. My little audio telemetry system is using Synchronous UDP, which is more typical for realtime data transfer. If something happens, just drop it on the floor and proceed. With TCP/IP you have extensive retry and error recovery that gets in the way of realtime data monitoring. > On Nov 15, 2025, at 06:33, Tim Bradshaw (as tfb at tfeb dot org) <[email protected]> wrote: > > On 14 Nov 2025, at 21:30, Tim Bradshaw <[email protected]> wrote: >> >> With this it seems to be able to do zero-copy (at Lisp level) I/O of raw vectors ... pretty quickly if they're on the same host. I need to do some proper benchmarks. > > On an M1 this seems to mean 'about 10GB/s' between two processes on the same machine. it varies a bit. This is for lots of big vectors. Smaller ones would have more overhead (I don't know what pinning a vector costs in terms of time, but that could be optionally raised out of the functions of course. > > I don't know how this compares with what the machine / network stack can do. > > --tim