Bring AnyData / DBD::AnyData back to work with modern DBI

Hi,

for a recent project we identified DBD::AnyData as best concept to do a client's job ;)
So there is a tuit to do the groundwork for bringing AnyData back to modern DBI interface around DBI::DBD::SqlEngine based drivers.

Last weeks I spent some time digging into AnyData itself to identify interfaces to touch for harmonization with the data_sources concept in DBI::DBD::SqlEngine (DBD::File). This mail is intended to share the results and present a concept for resurrection of the (more or less) dead module. For that reason, I CC'ed some people who got in touch with me over last years regarding AnyData and/or DBD::AnyData - to give them a chance to contribute.

At first the situation as it is: We (dbd-file-team, in this special case more or less Merijn and myself) identified most of AnyData and DBD::AnyData as being dead and nearly unusable in environments with modern Perl and up-to-date CPAN modules. The module is grown, bloated (no judgement for the time of writing), inconsistent and kind of self-contained (no reasonable API to outside). AnyData::Storage::TieHash is not a storage class, it's a miniature Tie::Hash::DBD with own query processing (parallel to DBI's SqlEngines and weird automatisms). I stop here to avoid starting a flame-war - the intension is to improve, not to blame.

So where is the future of AnyData?

From my point of view, upcoming AnyData / DBD::AnyData shall be reduced to the max. That means: no embedded adTie, no complex logic in frontend to deal with grown backends. Clean API for format-parsers, clean API for storage harmonized with DBI::DBD::SqlEngine::TableSource and DBI::DBD::SqlEngine::DataSource.

Consequently, upcoming releases of AnyData will depend on DBI. Format-Parsers will be written using DBD::CSV and DBD::DBM as guide (simple get_record, put_record etc. wrapper). To provide a tied hash again, DBD::AnyData will be bundled with AnyData (instead of two distributions in past) and Tie::Hash::DBD or Tie::DBI will be used.

adConvert, adDump and adExport are special cases of features already provided by SQL::Statement and will be re-implemented by using that functionality.

That all means, future API might be puzzled using roles to avoid strong requirements (Moo or Role::Tiny isn't decided yet) and unfortunately most of existing format-parsers in DarkPAN might require a rewrite. I hope the AnyData resurrection will help to reduce maintaining costs for future and apologize here and now for resulting extra effort when updating.

Best regards
-- 
Jens Rehsack
pkgsrc, Perl5
[email protected]
cpanid: REHSACK