Disgusting hacks [was Re: I wonder if I'm missing something...]

I haven't been following all this in detail, but there is a disgusting hack you can do in LW which I have, partly, working.

If you're dealing with socket streams, you don't care at all about data layout (which is likely fine if the two ends are the same), you only care about vectors which LW can pin, and you only use the socket streams for this and nothing else, then you can exploit the fact that a socket stream contains a socket which (if it's not some Java nasty) is, well, a socket.

And write(2) and read(2) / recv(2) know how write and read from sockets.

So the trick is to work out how big the data is you want to read or write, get hold of the FLI pointer for the vector (offset if need be) and then just vomit it out down the socket.

At the other end you use recv(2) to get the data into a suitable vector.

At the Lisp level this is as zero-copy as you can be.  There probably still is copying in the OS (it is possible to evade this, but I no longer remember how it's done and it is anyway likely very platform-dependent.

It's a disgusting hack because it's just poking inside the stream to get the socket.  For reading my current proof of concept also does not deal with any buffered data in the stream object although it does try to detect that there's data in the buffer and give up if so.

As I said I have a proof-of-concept code which half works: it will transfer a fairly big double-float vector (or any other suitable vector I am sure) but it then breaks on the next iteration.  I suspect this is because my server (which does the writing) is wrong but it may also be because, Unix being the OS made from the bits left-over from a number of better OSs, things like write(2) can just decide they're having a lazy day and they'll write some, but not all, of your data and ask you if you'd like to try again, please: I don't check for that.

I've lost interest in it really, but if anyone wants it I can send the current proof of concept.

The following function is useful:

(defun array-element-bytes (array &aux
                                  (element-type (array-element-type array))
                                  (cache (load-time-value (make-hash-table :test #'equal))))
  ;; How many bytes does an element take in an array.  Note that this
  ;; may return a rational < 1
  (or (gethash element-type cache)
      (setf (gethash element-type cache)
            (case element-type
              ((t)
               (error "No specialized array for ~S" element-type))
              ((character)
               (error "LW can't pin arrays of ~S" element-type))
              (otherwise
               (let ((a (make-array 9 :element-type element-type :allocation :pinnable)))
                 (with-pinned-objects (a)
                   (/ (-
                       (with-dynamic-lisp-array-pointer (p a :start 8)
                         (pointer-address p))
                       (with-dynamic-lisp-array-pointer (p a :start 0)
                         (pointer-address p)))
                      8))))))))

--tim

_______________________________________________
Lisp Hug - the mailing list for LispWorks users
[email protected]
http://www.lispworks.com/support/lisp-hug.html