RE: Re: Referencing a row in a two dimensional array

For now, the best way to get good performance in reading large blocks of
data is to allocate a foreign block of sufficient size and read into
that via foreign calls. You could use map-file if you want to read from
a file, and then use a loop to copy from there to a lisp allocated
array. Or you could just leave it in the foreign block and work with it
directly. Note that if you use map-file, and then modify the block, you
will modify the file (which may or may not be what you want).
READ-SEQUENCE is not currently more efficient than simply reading an
element at a time.

A better solution would be for me to implement pinned arrays i.e. blocks
of memory which do not move during GC. I have considered this for some
time, and my look into this in light of this discussion. They would
essentially be like foreign blocks, with a mark and sweep collector
cleaning them up, and could be accessed from lisp directly using normal
lisp functions. Or from foreign code.

You can't turn the garbage collector off. This was a deliberate design
decision. I also have not defined which functions allocate and which
don't. Therefore you can't really prevent allocation by steering clear
of functions which allocate (cons).

Because allocation can be so fast, it is close to the level of
efficiency of stack allocation. Consing is used inside many library
functions that you might not guess would use it.

The Begin-atomic and End-atomic assembler pseudo-instructions are
designed to allow small bits of code to be generated which leave some
lisp heap pointer untagged for one or more instructions. It would be
virtually impossible to prevent the code generator from ever generating
an untagged pointer, because some operations logically take more than
one instruction. It also allows some more advanced optimizations. The
garbage collector, if it sees that a thread is in the middle of an
atomic block, will give the thread a few cycles to get out of the block.
It is expected to get out of the block soon, so the GC can continue to
run. It is not intended that you use this to stop the collector, as you
would be basically halting all lisp execution. You could probably
implement a memcpy() type function which would copy from a foreign block
to a lisp byte array, or vice-versa, wrapped in an atomic block, using
assembler. I may write one of these and I will forward it if so.
Actually if you are careful the atomic block should not even be
necessary, provided no registers point inside the lisp heap block during
the course of execution. If a gc occurs, then the index registers will
be updated appropriately.

It is true that the kernel uses a critical section on the garbage
collector to prevent multiple threads from invoking the collector at the
same time. However it is not intended as a way to stop the collector for
some other purpose.

There are some pretty tight rules regarding how the collector and
execution model work, but if these are not strictly enforced things go
downhill fast (believe me). 

I very much understand the need for a fast READ-SEQUENCE that is
optimized for bytes. It doesn't exist yet (unless Pavel or someone has
written it already). I will try to prioritize working on this, as it
would be extremely useful to me as well. Like I say, for now I would
resort to foreign heap blocks for this.

Pavel, if you wish to share some code with someone involved with this
discussion I have no problem with that. I just ask that you restrict the
persons getting copies of your changes to those in this group and make
sure they understand it is your own modified version (i.e. don't post in
a public place on the internet). I would like a copy as well. I
appreciate what all of you have done and continue to do to improve
Corman Lisp, and I realize I am not able to keep up with all the things
people need.

Roger
-----Original Message-----
From: Chris Double [mailto:[email protected]] 
Sent: Saturday, March 01, 2003 6:34 PM
To: [email protected]
Subject: Re: [cormanlisp] Re: Referencing a row in a two dimensional
array


On Sun, 02 Mar 2003 14:08, JP Massar wrote:
> If you can turn the GC off at all, then why can't you turn it off for 
> an indefinite amount of time, as long as you neve cons (i.e., allocate

> memory), and insure that other threads can't run?

I forget the exact reasons, I'm sure Roger can expand on it. The flag
for 
turning the GC off is an assembler register I think. So you can 'clear'
and 
'set'. If you set it and call another function that clears and sets it
then 
on return from that function you've got problems.

Also, when the GC wants to run it scans all threads to check if that
register 
is clear (or set) to determine if it can pause the thread to run the GC.
Long 
running operations that turn it off will result in inefficient GC
perhaps?

Chris.
-- 
http://www.double.co.nz/cl


To unsubscribe from this group, send an email to:
[email protected]

 

Your use of Yahoo! Groups is subject to
http://docs.yahoo.com/info/terms/ 



------------------------ Yahoo! Groups Sponsor ---------------------~-->
Get 128 Bit SSL Encryption!
http://us.click.yahoo.com/LIgTpC/vN2EAA/xGHJAA/SyjtlB/TM
---------------------------------------------------------------------~->

To unsubscribe from this group, send an email to:
[email protected]

 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/