RFC: developing DBD::SQLite

Hello,

So, out of my un-paid projects, my promise to take over release management of 
DBD::SQLite (from the still incommunicado previous owner) has now come to the 
front of my queue (now that Set::Relation 0.9.0 is out), so I'm now starting to 
think about it in detail and get to work over the next week or two.

In that vein, the first and only major design change I intend to make right from 
the start is to stop bundling the SQLite library with the DBI driver and so the 
driver will have that library as an external dependency.

While one of the selling points of DBD::SQLite versus other DBI drivers in the 
past was that it came with everything you need, with the advent of a single file 
"amalgamation" library being provided standard from sqlite.org, as well as the 
increasing availability of the SQLite library as its own shared system library 
install, I figure it isn't too difficult now for users to either obtain the 
library separately or use the one that came with their system, or the 
DBD::SQLite installer could automatically download it similar to how some other 
projects download their dependencies (Rakudo Perl 6 can download Parrot for 
example); so I don't think the ease of use of DBD::SQLite is diminished 
significantly by it no longer bundling the SQLite library.

On the other side, there is a lot of benefit gained from not bundling.  For one 
thing the size of the distribution as well as the source control is cut down 
significantly, since the DBI driver alone is orders of magnitude smaller.  For 
another thing, occasional needs to update for compatibility aside, DBD::SQLite 
will always be up to date with the newest SQLite library; users just update the 
library and possibly recompile the DBD::SQLite they already have.  And so 
DBD::SQLite won't need to be updated as often; it would just need updates to 
address incompatible changes to the SQLite library, or to fix bugs in or update 
features specific to DBD::SQLite itself.

Another quasi-change is that DBD::SQLite will be designed to work specifically 
with the amalgamation version of the source files only, not the pre-amalgamated 
versions; I say quasi-change since Audrey Tang already did the work to convert 
DBD::SQLite to work this way, in the separate ::Amalgamation distro.

Compatibility-wise, my DBD::SQLite will endeavour to work with all versions of 
SQLite 3.y.z, though note that only 3.4.0 for which the amalgamation file was a 
distinct download on sqlite.org (and 3.3.14 or so was the first that 
amalgamation was a make target).  Or more specifically, I only plan to test with 
the latest SQLite source library available at the time (3.6.11 currently), as 
well as probably whatever version comes with Mac OS X Leopard.  Supporting older 
versions will happen as I get advocates or testers for them.  I also won't 
explicitly drop support for any Perl or DBI versions that the current 
DBD::SQLite supports, but I only intend to test it with the latest DBI and Perl 
5.8.x+.

A minor change is I will start out with using 3-part versions and have a 
dependency on version.pm 0.74, which is bundled with Perl 5.10.x and an easy 
install otherwise.

Now a specific question for you:

First assume the new DBD::SQLite can look in at least 3 places for a SQLite 
library to use, which are:  1. An amalgamation file that the user explicitly put 
in the distro directory (or that was similarly slipstreamed into a copy of the 
distro maybe by some OS package manager); 2. A SQLite system shared library that 
was installed either as part of the OS or later by a user; 3. Go and 
automatically fetch a copy of the latest amalgamation file from sqlite.org, 
similarly to how Rakudo Perl 6 can go fetch a copy of its Parrot dependency from 
the 'net.

Now assuming that, changeable config options aside, there is an automatic 
default order that these alternate sources will be used by a hands-free 
CPAN/CPANPLUS/etc install, I ask you what options should be used and in what 
order?  Now I think we probably can all agree that if choice #1 is satisfied 
(the user explicitly put a file to use in the distro dir or a packager did), 
then it is clear-cut to try to use that or die trying.  What is less clear cut 
is whether the next automatic default action should be to use an already 
installed SQLite shared library or whether to download the latest from sqlite.org.

I'm thinking that between #2 and #3, that #3 (get the latest) should always be 
the default choice, and that users have to explicitly override, such as by 
setting an environment variable, or whatever already happens with DBD::SQLite, 
to use the system one instead.

So in fact the real question then is whether I should stay with the precedent 
existing with the current DBD::SQLite, which effectively is download from 
sqlite.org by default, or whether I should change this default.

Any thoughts on that matter, or on anything else I raised?

And while I suppose this is technically a fork, I'll still be just using the 
name DBD::SQLite so my versions are picked up automatically by users of the 
previous.

Now as for version numbering, I decided already to not use 4-part versions like 
eg 3.8.11.0, but rather just use 3.x.y_NNN format for my releases, with the X 
and Y values following my own preferred pattern of Y for feature changes and Z 
for bug fixes, but that since prior releases are already production status, I'll 
also use the _N suffix convention for my initial or experimental releases and 
just release without _N when testers report no show-stoppers, similar to how the 
likes of DBD::Pg or Moose etc are released now.

So I'm wondering what would be the least contrived looking numbering scheme in 
light of starting with _N, eg; would 3.0.0_001 be the best for dev release start 
and then either 3.1.0 or 3.0.1 (depending on whether features change from the 
_N) be the first non-dev release, or should I start the dev releases in perhaps 
a more contrived 2.y or 1.y space?

Now I recall that someone had previously proposed making the SQLite library an 
external dependency when my initial proposal didn't.  If they or anyone else 
wants to speak up with specific advice or patches etc I'd appreciate it.

Also my DBD::SQLite will be in a public Git repository, either on utsl.gen.nz or 
github or both, so I can receive patches by pulling from someone's fork.  Its 
not up yet though.

Thank you in advance for any feedback.

-- Darren Duncan