Re: Enhancements to Pure-Perl DBD's (primarily DBD::File)

On Tue, 18 Sep 2012 02:12:18 -0700, Darren Duncan
<[email protected]> wrote:

> Jens Rehsack wrote:
> > today I got some people in irc://irc.perl.org/#dbi to discuss planned
> > extension to DBD::File.
> > 
> > There were several goals to reach:
> > 1) add readonly mode to DBI::DBD::SqlEngine
> >    ==> will be solved by adding a new attribute sql_readonly to
> >        DBI::DBD::SqlEngine and if required fix DBI::SQL::Nano and
> >        SQL::Statement and bump SQL::Statement requirement in *::Nano
> 
> I would recommend doing the complement, backwards-compatibility concerns aside.
> 
> Make read-only the default behavior and have a new attribute sql_writable 
> instead.  If the goal is to make it possible for safer behavior, that should be 
> the default.  Then people don't "accidentally" make changes when they didn't 
> mean to.

I would not. That is VERY counter-intuïtive. All (other) DBD's are
read-write by default, as it should be.

> If backwards-compatibility is the only reason not to do that, you can phase it 
> in over time maybe in a similar manner to how strictures were in Perl; eg, if 
> someone explicitly requires a version number that is greater than X, then 
> readonly is the default; if they have no explicit version requirement, then 
> something backwards-compatible is the default.

No, read-only support is only added to make implementing streams as
data-sources easier. Maybe just for the thought-process, but still.

Jens and I talked this through when imagining a file like
database.tar.gz being opened using Archive::Tar, which contains many
CSV files (the tables). A good example for a read-only data-source.

Jens then proclaimed it would probably not bee too hard to extend the
API to allow writable streams, which I doubted, but discussion went on.

> > 2) add support for other I/O layers to DBD::File
> >    a) add support for streams (PerlIO)
> >    b) add support for other kind of fetch_row/push_row processing
> 
> Does the current version work with files in a stream fashion,

No, DBD::File *always* uses <c>open</c> to open a data-source. Maybe
that is why it is called DBD::File.

> such as only reading in enough at a time that it needs, or does it
> read the whole file before doing anything?

That depends on the implementation in the DBD that extends DBD::File
e.g. DBD::CSV reads the complete file at once.

> Regardless, streaming is great for scalability.  It means you can have 
> many-gigabyte files and be able to work with them.  And security is helped as it 
> isn't so easy for someone to DOS you by giving you a huge input.

Yes, one of the thoughts we discussed

-- 
H.Merijn Brand  http://tux.nl   Perl Monger  http://amsterdam.pm.org/
using perl5.00307 .. 5.14   porting perl5 on HP-UX, AIX, and openSUSE
http://mirrors.develooper.com/hpux/        http://www.test-smoke.org/
http://qa.perl.org   http://www.goldmark.org/jeff/stupid-disclaimers/