Re: Upcoming release 2.11

Moin Edi,

Edi Weitz schrieb:

>On Wed, 06 Apr 2005 10:03:57 +0200, Heiko Kirschke <[email protected]=
e> wrote:
>
> =20
>
>>Second, to me it looks as if Plob should use its own underlying
>>persistent heap. There are several options to this:
>>
>>1. Abonding the mixed C-/LISP-approach of Plob, coding the
>>persistent heap in LISP. Would mean a major redesign and cost a lot
>>of effort.
>>   =20
>>
>
>But I seriously wonder if it's not worth it.  A pure CL solution will
>definitely be easier to debug and port, and it'll be easier to deliver
>applications based on it.  Plus, as Plob is an open source project, we
>should not forget that it's most likely much more fun for Lisp hackers
>to work on Lisp code than to work on C code... :)
>
>FWIW, I'm asking myself why the Lisp community seems to agree that the
>lowest levels of a CL persistent storage layer cannot be implemented
>in CL itself.  Facts:
>
Well, my reasoning is based more on, hmm, thinking about effort than
thinking that this would not be technically possible with a pure-CL
solution.

>1. Plob is currently based on a C library and what you and others have
>   proposed as replacements in this thread are also C/C++ libs.
>
>2. AllegroStore, as offered by Franz, is based on ObjectStore.
>
>3. Franz tries to replace AllegroStore with AllegroCache which is
>   based on Berkeley DB.  (I've seen Jans Aasman demoing it in
>   Amsterdam and he'll most likely repeat his talk at the European
>   Common Lisp Meeting at the end of this month.  You can also read
>   Bill Clementson's blog entries about it.)
>
>4. Elephant is based on Berkeley DB.
>
>Why is that?  I understand that this is something that's hard to do
>right but is it so hard that even the largest CL vendor can't do it?
> =20
>
Since Franz is expert in Lisp Systems, I guess they wanted to buy some=20
database instead
of coding it from scratch, as I assume also for the reason not having=20
the efforts to code a
complete database system on their own. Expectations on a working=20
database system
w.r.t to its ACID properties tend to be very high.

>I don't expect a solution that's competitive with, say, Berkeley DB
>from the very beginning, I'd be happy with something that's slower
>than a pure C solution my a small constant factor.  I rarely have to
>deal with billions of objects.
>
>I also understand that a Lisp solution will probably involve
>implementation-dependent and OS-specific code.  Well, so be it.  At
>the moment you need the MOP and the FFI anyway, so the situation is
>the same.
> =20
>
Yes, indeed. And each new Lisp release introduces much more #+: es and=20
#-: es into
Plob's code ...

>>For option 2., the POSTORE API is not that complicated. There are
>>functions for allocating a persistent record with a quite simple
>>structure (references to other persistent records and values [i.e.,
>>no references]), access functions to these records by their `object
>>id', and some administrative functions like flushing, creating a new
>>persistent heap etc. You find the interface in the SH_* functions
>>declared in plob-2.10/src/include/postore.h. All other
>>databasespecific stuff (transactions, btrees etc.) has been
>>implemented in Plob anyway in the layer above the persistent heap.
>>   =20
>>
>
>I'm confused here.  Let me summarize what I believe to understand and
>please correct me where I'm wrong:
>
>1. POSTORE offers efficient storage of (untyped?) records and is able
>   to maintain and follow references between these records.  It also
>   offers collection of unreferenced records.
> =20
>
Yes to a large degree. On POSTORE level, the records are untyped but=20
structured. There is
a structure imposed on each record, one half being references to other=20
records, the other half
being uninterpreted values (see also chapter 7.4 of diplom.pdf provided=20
within the Plob
distribution).

>2. POSTORE does not offer transactions or btrees both of which have
>   been implemented in Plob instead.
> =20
>
Yes, and the POSTORE records are typed in the layer ,,above'' POSTORE.

>3. Berkeley DB offers btrees and transactions but AFAIK no references
>   between records and certainly no garbage collection.
> =20
>
Yes. This could be added on top of  the Berkeley DB layer as an=20
additional layer.

>How does that relate to your plan to replace POSTORE with Berkeley DB?
> =20
>
It would mean to use Berkeley DB as a kind of persistent heap, too, i.e.=20
impose a structure
on Berkeley DB's notion of database records. For Berkeley's additional=20
functionality of
btrees and transactions this would mean that either the mechanisms of=20
Plob are used
and Berkeley's would not be used at all, or that the Berkeley DB=20
mechanisms are passed
through all layers up to the topmost API layer of Plob and remove these=20
functionality
from the pure Plob layers.

>And, while I'm at it: Are transactions and btrees currently
>implemented in C or in Lisp?
> =20
>
Completely C: in splobheap.c and splobbtree.c

>>Since my time for working on Plob is rather constrained, I would
>>prefer option 2.
>>   =20
>>
>Well, I didn't necessarily expect /you/ to do all the work.  I was
>contemplating to do something myself and for that I'm trying to sort
>out what has to be done.
> =20
>
You're welcome! If you or someone else could show up with a solution of=20
providing
some low level persistent layer in LISP, I would be happy to use it.=20
Again, I'm thinking
in terms of effort, i.e. to create a good working solution in a moderate=20
amount of time,
that is my reason d'etre. I see no technical obstacle for a pure LISP=20
solution, and also
would prefer this kind of solution. IMHO, technical details on this can=20
be discussed
offline.

(There is a minor technical reason I had in mind when using C for the=20
lower layers: The
idea is that C is a kind of normed assembler, and that almost any other=20
programming
language is able to connect somehow to C (i.e., by some kind of foreign=20
function calls
resp. ,,native'' interface, e.g. Java's JNI). So, when using C for=20
Plob's lower layers,
the upper layers could have been done in some other programming language=20
besides
LISP. Since this has never been carried out, it makes sense to forget=20
about this idea.)

Viele Gr=FC=DFe,
Heiko


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=3D6595&alloc_id=3D14396&op=3Dclick