ANN: refdb-0.9.9 released

Hi all,

today I've uploaded the RefDB version 0.9.9 sources, along with
updated or new Perl modules required by this release. The
documentation and DTD files available on the web page were updated as
well, and the web page itself also received some polish. Please visit:

http://refdb.sourceforge.net

for general information, and 

http://sourceforge.net/project/showfiles.php?group_id=26091

to download the files.

The changes compared to the previous release are quite substantial and
are probably the hugest between any releases so far. Besides a
boatload of internal changes that simplify future developments, there
are a couple of new or improved features. There is a much nicer and
easier-to-use PHP web interface written from scratch, MODS support,
SRU support, namespace support in all XML formats, raw bibliographies,
and support for DocBook V5.0 and TEI P5, to name a few. Please see the
detailed list of changes below for further information.

I recommend to consult the UPGRADING file before installing the new
release. Both your reference data and custom bibliography styles may
require some manual fixes to work best with the new version.

As there were *so* many changes and new features, I'd be surprised if
there was not a single bug. Please report problems, suggestions and
rants to this list.

regards,
Markus


Bugfixes:

- raw bibliographies were fixed again

- a bug in the tokenizers used by refdbd and by the clients to parse
  command line arguments has been fixed. Now escaped characters should
  be treated as such in all cases.

- fixed the missing author type in part authors imported from risx

- fixed the style export on OS X (PPC)

- fixed the publication year output in RIS export if the otherinfo
  field is empty

- crash during bibliography creation on OS X was fixed

- unknown reference data types are now changed to "GEN" when importing
  risx data as well

- abbreviated journal names which use periods without spaces are now
  imported properly

- config.rpath was added to the tarball to avoid configure errors on
  particular OSes

- upgrading the main database no longer creates duplicate entries in
  t_meta when using PostgreSQL as the database engine

- fixed missing permissions after upgrading the main PostgreSQL database

- the preconfigured locations of log and pid files now honor the
  prefix (usually /usr or /usr/local)

- fixed a table locking error if a risx dataset is skipped during
  import

- special characters are now properly converted to XML entities even
  in long texts with lots of these characters

- now purges "(", ")", and "?" from citation keys

- now checks the reference types before adding them to the
  database. Unknown types are changed to "GEN".

- a sanity check in the RIS import code prevents segfaults if risx
  data are accidentally fed to it.

- risx datasets which are accidentally added as RIS datasets no longer
  crash refdbd.

- a possible client/server hiccup caused by checkref reports of a
  particular size was fixed.

- the handling of the (entirely broken by design) issue, edition, and
  volume fields in a variety of reference types has been improved

- when checking risx datasets the tables are now properly unlocked if
  a dataset is skipped


Improvements:

- the en2ris script now uses UTF-8 as the default output encoding

- boatload of improvements of the PHP web interface, among these is
  the tag cloud display of authors, keywords, and periodical names in
  the search results.

- the handling of the AV field has been cleaned up. As of this
  version, the AV field should only contain a physical location,
  i.e. either a description of the location, or a call number. The
  formerly applied PATH: kludge to manage links to local copies of PDF
  files is no longer supported. To store user-specific locations of
  PDF files, use the L1-L4 fields and start the URL with the "file://"
  protocol prefix. These links are assumed to point to local copies
  owned by the user. If the URL uses a relative path, i.e.
    "file://offprints/Miller2002"
  instead of
    "file:///home/markus/offprints/Miller2002"
  (note the additional root slash in the latter URL), the same pdfroot
  mangling is applied as was previously to the AV  - PATH: paths.
  Please see the UPGRADING file for some hints how to efficiently
  migrate existing AV field contents.

- in order to reflect the above change in risx datasets, the risx dtd
  was adapted accordingly. The <link> element is now also allowed in
  <libinfo>. Although the DTD can't enforce this, the pubinfo/link
  elements are supposed to hold publicly accessible URLs of offprints,
  images and such, whereas the libinfo/link elements should receive
  only user-specific local paths to offprints.

- if a character encoding conversion is necessary during data import
  or data export, refdbd now uses the transliteration facility of iconv
  for characters which cannot be mapped to the target encoding, a
  problem which often arises when exporting UTF-8 data to European
  encodings.

- the new PHP web interface was thoroughly overhauled to cope with
  whitespace, apostrophes, and regular expression characters in
  keywords, author names, and journal names. Adding references now works
  from risx data too, and the feedback about the success of adding a
  file has been improved. Adding data from the web form was improved too
  in that empty fields are left out, author name and keyword lists are
  appropriately tokenized, and addref or updateref is used whenever
  appropriate. The code now makes sure to use a unique temporary filename
  which is deleted again afterwards, avoiding clutter in your /tmp. The
  field descriptions of the Add reference form was overhauled too to better
  reflect the contents depending on which reference type is used. The month
  selector now creates a numeric month to comply with the RIS format's
  requirements.

- added a getax command to retrieve authors of any level

- added namespace handling for DocBook and TEI stylesheet driver
  files. TEI P4 and DocBook V4.x files are now always processed with the
  non-namespaced stylesheets, whereas TEI P5 and DocBook V5 files are
  handled by the stylesheets with namespace support. In contrast to the
  stock DocBook stylesheets there is no auto-detection whether or not
  the document uses namespaces.

- TEI P5 cooked bibliographies are now directly generated by refdbd
  instead of transforming them from DocBook bibliographies

- refdbib now optionally retrieves raw bibliographies based on a
  citationlistx input document. This allows using RefDB as a data
  source for SGML/XML documents even if RefDB's citation and
  bibliography formatting capabilities are not desired or applicable

- the PHP web interface has been replaced with a new implementation
  from scratch (thanks to Dan O'Donnell and Dominik Reusser).

- added config files for runbib and refdbjade to pass additional
  options to (open)jade, especially to increase or disable the error
  limit

- added a standalone SRU server for personal use or for testing purposes

- bibliography styles now have elements to record the style author, a
  description, and an URL that points to the source of the
  bibliographic rules.

- added a new legal bibliography style contributed by Jeremy Malcolm

- DocBook and TEI XML output is now indented and can optionally be namespaced

- DocBook XML V5.0 support

- the internal representation of authors and titles is now based on
  the three-level library system (part, publication, series) instead of
  on the once-and-for-all-screwed RIS definition. This makes both the
  programmer's life and the style author's life quite a bit easier

- the internal representation of the M1-M3 fields was improved to
  simplify data access. The contents of these RIS fields is now stored
  in separate fields which reflect the purpose in context of the
  respective reference type

- citation styles now support both authors and editors in monographic data

- citation styles now support the replacements of the M1-M3 fields,
  i.e. stuff like conference location, sender and recipient email
  addresses, class codes, running time and so on

- the output of adduser/deleteuser was cleaned up if no hostname is
  available

- the format specifier NX of the getref command no longer retrieves
  all extended notes associated with a particular reference, as these
  would always include the users reference lists. If you positively want
  those lists, use ALL or NXNL instead.

- the citestylex.dtd now supports different formatting options for
  authors (TEXT) and editors (TEXTED). The styles shipped with RefDB
  were updated accordingly.

- all backends were updated and now export the contents of the RIS
  fields M1-M3 as far as possible

- two new citation styles were added (Tissue_Eng. and Biomaterials)

- the bibliography sorting order is now case-insensitive and takes
  into account missing authors, using the title for sorting in this
  case

- the MySQL reference database schema now uses TEXT instead of BLOB to
  make use of the improved character encoding and collation features of
  newer MySQL versions

- the new refdbc:countref and refdbc:countnote commands are equivalent
  to the getref and getnote commands, respectively, except that they
  do not actually retrieve the reference or note data. Instead, they
  return the number of matching references and notes, respectively.

- the query language supports additional relational operators: <=
  (less than or equal), >= (larger than or equal), <> (not like).

- there is a new backend for MODS output, an XML format maintained by
  the Library of Congress.

- the XML output (risx, mods, xnote) now supports namespaces.

- the -n switch of getbib is no longer supported (it was hardly ever
  used anyway). refdbd now sends a stylespec no matter what.

- the getnote/countnote commands now also support query strings of
  unlimited length, just like the getref/countref commands.

- the adduser/deleteuser commands now support two classes of users:
  read-only users and read/write-users. The former is a simple way to
  generate a database account for users accessing your databases through
  SRU or via the web interface without permitting write access to
  your databases for everyone.

- the new CGI script refdbsru provides an optional SRU (Search and
  Retrieve via URL) interface to your databases.

- the checkref command can now be limited to check individual fields
  or combinations of fields.

- improved import of tagged Pubmed data (med2ris)

- digital object identifiers (DOI) are now harvested from Pubmed data
  and are stored properly. DOIs are presented as links (via
  dx.doi.org) by the (x)html backend.


Backwards-incompatible changes:

- the -i option of refdbjade was changed to I (capital i) to make it
  the same as in runbib (which uses -i for a different
  purpose). runbib in turn now uses -s instead of -I to skip the ID
  extraction step, and -I instead of -m to set include options


-- 
Markus Hoenicka
[email protected]
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/