Re: JDBC persistence

Hi John,
thanks for your contribution.
I am ready to commit the files to the trunk, however I still do not see 
your name on the list of jca-approved contributors, so I have to wait. 
Please let me know as soon as you sign the JCA and your name appears on 
the list.
Now to your questions:

> OK, I have submitted the first cut of the code (see issue 38403).  In 
> some areas I was working in the dark, so I have quite a few TODO issues 
> at the end of the package.html file; guidance on what needs to be 
> implemented would be appreciated.

OK, here are some suggestions regarding your todo's (my comments are 
prefixed by [MaM])

1) JdbcStorageFactory currently reads System.getProperties() directly, 
rather than using the properties passed to createStorage(). The reason 
is that MDR doesn't know about the JDBC-specific properties. Once it 
does, JdbcStorageFactory should stop using System.getProperties().

[MaM] System properties prefixed by "MDRStorageProperty." are passed to 
the createStorage method. So all you need to do is to prefix your 
properties by MDRStorageProperty - e.g. 
"MDRStorageProperty.org.netbeans.mdr.persistence.jdbcimpl.url" and it 
will automatically be passed in the storage properties map under 
"org.netbeans.mdr.persistence.jdbcimpl.url" key.

2) I wasn't clear on the semantics of unique-valued MultivaluedIndexes, 
and memoryimpl and btreeimpl didn't provide a consistent answer. The MOF 
boot sequence resulted in some attempts to insert duplicates.

[MaM] That's strage - bootmof should not create unique-valued indexes. 
If it is really the case, it is probably a bug in MDR and we should 
investigate it. But currently it is of low priority.

3) I didn't strictly follow the spec for non-unique values: 
remove(key,value) removes all, not just the "first" one (since "first" 
has no meaning for an unordered index), and the same goes for the 
iter.remove() equivalent.

[MaM] This is an incorrect behavior. You are right that order has no 
meaning in unordered indexes, so you don't need to care about which 
occurrence of the element you remove, however count does have a meaning, 
so you cannot remove all of them! If element "a" occurs twice in the 
index and I remove it only once, I expect one "a" will still be in the 
index.

4) I did not implement keySet() and values() for JdbcPrimaryIndex 
because in my testing I never saw them get called. Nor did I implement 
queryByKeyPrefix() for any of the indexes, or fail-fast iterators anywhere.

[MaM] This may be a problem. In some of our internal project containing 
custom implementations of JMI object we are touching some of these 
methods (e.g. queryByKeyPrefix). However, it is up to you to decide 
whether you want or do not want to implement it. I agree that these 
methods will probably not be used in 99.9% of MDR clients.

5) Due to (I'm guessing) an oversight, MDRCache.replace() is not public 
as it should be. To work around this, I injected {@link 
org.netbeans.mdr.persistence.btreeimpl.btreestorage.AccessHack} to sneak 
in the required access.

[MaM] I don't think this was an oversight. MDRCache was not written with 
goal of building a general purpose cache for different storages in mind. 
It is an implementation specific thing of b-tree storage. There are 
probably several things that could be factored out of the b-tree storage 
into some kind of a storage-independent utilities. We are open to that 
if it does not break anything or does not have a negative effect on 
performance.

> If you want to try it out, the easiest back-end to use is hsqldb 
> (http://hsqldb.sourceforge.net) since it's pure in-process Java, and 
> it's what I used for most of my testing.  I also tested with PostgreSQL 
> (for which you have to specify BYTEA for the STREAMABLE datatype). 
> Others may work out-of-the-box, but will probably require a setting for 
> the STREAMABLE datatype.

Sounds good. I admit I haven't had time to try it out yet. Hopefuly I 
will be able to do it over the weekend.

> Also, I'd like to run through the MDR test suite (especially the 
> performance tests).  Is there an easy way to apply system property 
> settings to all of the tests?  The test framework is quite heavyweight 
> so I didn't see an obvious place to start.

We need to investigate this and try to make that possible (if it isn't).
I am very interested in running the whole JMI TCK on the MDR backed by 
your storage. It includes about 1000 different tests. I will do it as 
soon as I get your storage working and let you know what the results were.
Martin