Re: lost in the devel--1.3 branch

Frédéric Gobry wrote:

>> At this point I can only assume I am on the wrong track and that I 
>> must be missing something obvious. Any ideas? I'm using Python 
>> 2.3.4... nothing fancy as far as I know.
>
>
> You're not missing sth obvious : core-api contains a few files 
> resulting from long discussions with Peter, which led to two
> separate attempts for some functions. Therefore, there are a few 
> classes and their associate unit tests that are not
> correctly integrated in the framework. Unfortunately, I did not 
> maintain them, and now they produce unit test errors.
>
Ok, no problem, I won't dwell too long on the tests then. Even though 
core-api is in its early days and is likely to be a moving target for 
some time yet, I really like the look of it so far and have decided to 
give it a red hot go. I expect to blunder blindly for a while and to ask 
some pretty dumb questions, so please bear with me.

In order to become better aquainted with core-api, I started knocking 
together a short python script to import a bibtex database and dump the 
bib info on stdout. What I came up with was;

> #!/usr/bin/python
>
> import os, sys, string;
>
> from Pyblio.Importers import BibTeX;
> from Pyblio import Store, Schema;
>
> bibFile = "test1.bib";
> xmlFile = "test1.xml";
>
> dbFormat = Store.get("file"); # the core-api database format?
> dbSchema = Schema.Schema("./myschema.xml"); # the core-api database 
> schema?
>
> db = dbFormat.dbcreate(xmlFile, dbSchema);
>
> doc = Store.TxoItem();
> doc.names['C'] = "article";
> db.txo["doctype"].add(doc); # only support article citations for now
>
> parser = BibTeX.Importer();
>
> parser.parse( open(bibFile), db);
>
> iter = db.entries.itervalues();
> while 1:
>     try:
>         cite = iter.next();
>         for key in cite.keys():
>             print "%s : %s" % (key, cite[key]);
>         print
>     except StopIteration:
>         break
>
> Store.get("file").dbdestroy (xmlFile, nobackup = True);

I started with simple.bib and schema.xml from tests/ut_bibtex/ and this 
worked ok. I then tried it on a small bibtex database containing a 
couple of my actual @article citations. Of course it failed miserably.  
I added entries to myschema.xml for the volume = {}, pages = {} and 
month = {} fields in the bibtex citations;

> :
> :
> <attribute id="volume" type="text">
>  <name>Volume</name>
> </attribute>
>
> <attribute id="pages" type="text">
>  <name>Pages</name>
> </attribute>
>
> <attribute id="month" type="text">
>  <name>Month</name>
> </attribute>
> :
> :
>
and this seemed to take care of things and my citation data was dumped 
on stdout.

Having acomplished this, I have the the first inklings of a picture in 
my mind of how the core-api is probably intended to be used. If you have 
read this far, then I will really push my luck and run a few things past 
you for confirmation or correction.

It seems that:

1. The key entities in the core-api are these database objects. All 
bibliographic citations exist as records in a database.
2. The fields in a given database are determined by the schema with 
which it was created (via the dbcreate( ) method).
3. The schema is typically read from an xml document.
4. Databases may be saved on disk in a number of formats (I used "file", 
which produced xml, but garlic uses "bsddb").
5. The core-api also defines an abstract Record class which provide an 
abstraction to simplify the manipulation of individual records, 
irrespective of the corresponding database schema or the format of the 
underlying database.

Am I on the right track here? I havn't really looked into things much 
further yet, but presumably an application could contain more than one 
database simultaneously and individual records could be moved from one 
to another, whole databases could be merged etc. Is is up to individual 
applications to provide the database schema(s) and the concrete Record 
classes? 

Getting back to my simple example script above, is there a way to have 
the bibtex importer simply skip fields in the bibtex entries for which 
there is no corresponding entry in the core-api database schema? For 
example, if I wanted to compile some statistics based on a collection of 
bibtex citations, I might only be interested in say the first author, 
the title and the year of publication of each citation. I could define a 
schema containing only those fields and the bibtex importer would 
(should?) silently discard everything else, like journal names, volume 
and page numbers etc.

Along a similar line, what happens when the schema defines an attribute 
for which there is no bibtex analog? For example, if I wanted to sort my 
citations into categories, I would be tempted to add a category 
attribute to the schema something like that used by garlic.  Any 
citations I import from a bibtex database obviously don't include a 
category field, so I would like them to be placed into a default 
"unsorted" category. Maybe the way to do this is with a <default> tag in 
the database schema?

Thats plenty for now,

e.


-------------------------------------------------------
This SF.Net email is sponsored by: InterSystems CACHE
FREE OODBMS DOWNLOAD - A multidimensional database that combines
robust object and relational technologies, making it a perfect match
for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8