Re: Bring AnyData / DBD::AnyData back to work with modern DBI

Am 13.11.2014 um 14:37 schrieb Tim Bunce <[email protected]>:

> My impression was that there was a risk of breaking existing users apps.
> If that can be avoided then great, and keeping the name would be best.

Neither, nor :(
They're already broken.

The intension is, to pick them up and lead them to new meadow areas without broken module(s) :)

Jens

> Tim.
> 
> On Thu, Nov 13, 2014 at 07:12:33AM +0100, Jens Rehsack wrote:
>> 
>> Am 12.11.2014 um 16:14 schrieb Tim Bunce <[email protected]>:
>> 
>>> Perhaps the module name should be changed.
>> 
>> Why? Let me just talk about the reason to keep and the consequence to change.
>> 
>> Reason to keep: People continuous ask me about AnyData / DBD::AnyData and send me fiddly patches hacking in the module, complain about working around compat issues etc.
>> So what I currently know: every AnyData / DBD::AnyData user makes adoptions to the projects.
>> 
>> Consequence to change:
>> My requirements are easy: I need a DBD scanning a directory for files, opening the files and return the file name plus the :, separated fields (some optional, some key-value pairs) for each line. Hacking a new DBD would mean to me: clone DBD::CSV and adopt the parser.
>> 
>> Why did I decide for AnyData? Because the idea behind AnyData matches the requirements I have. It will be easier to find another consultant having knowledge about DBI and (new) AnyData than DBI, private DBD and the patch-supporting pkg-manager we'll use to add private prefix :)
>> Once we start local CPAN module patches, the motivation to contribute instead of hacking locally is reduced significantly (typical project flow - once a direction is chosen, it's fixated ...)
>> 
>> Beside the argument to keep an existing (working!) API for people who using it (which practically doesn't exists for AnyData/DBD::AnyData), why I should do a new DBD?
>> 
>> Jens
>> 
>>> Tim.
>>> 
>>> On Wed, Nov 12, 2014 at 10:33:20AM +0100, Jens Rehsack wrote:
>>>> Hi,
>>>> 
>>>> for a recent project we identified DBD::AnyData as best concept to do a client's job ;)
>>>> So there is a tuit to do the groundwork for bringing AnyData back to modern DBI interface around DBI::DBD::SqlEngine based drivers.
>>>> 
>>>> Last weeks I spent some time digging into AnyData itself to identify interfaces to touch for harmonization with the data_sources concept in DBI::DBD::SqlEngine (DBD::File). This mail is intended to share the results and present a concept for resurrection of the (more or less) dead module. For that reason, I CC'ed some people who got in touch with me over last years regarding AnyData and/or DBD::AnyData - to give them a chance to contribute.
>>>> 
>>>> At first the situation as it is: We (dbd-file-team, in this special case more or less Merijn and myself) identified most of AnyData and DBD::AnyData as being dead and nearly unusable in environments with modern Perl and up-to-date CPAN modules. The module is grown, bloated (no judgement for the time of writing), inconsistent and kind of self-contained (no reasonable API to outside). AnyData::Storage::TieHash is not a storage class, it's a miniature Tie::Hash::DBD with own query processing (parallel to DBI's SqlEngines and weird automatisms). I stop here to avoid starting a flame-war - the intension is to improve, not to blame.
>>>> 
>>>> So where is the future of AnyData?
>>>> 
>>>> From my point of view, upcoming AnyData / DBD::AnyData shall be reduced to the max. That means: no embedded adTie, no complex logic in frontend to deal with grown backends. Clean API for format-parsers, clean API for storage harmonized with DBI::DBD::SqlEngine::TableSource and DBI::DBD::SqlEngine::DataSource.
>>>> 
>>>> Consequently, upcoming releases of AnyData will depend on DBI. Format-Parsers will be written using DBD::CSV and DBD::DBM as guide (simple get_record, put_record etc. wrapper). To provide a tied hash again, DBD::AnyData will be bundled with AnyData (instead of two distributions in past) and Tie::Hash::DBD or Tie::DBI will be used.
>>>> 
>>>> adConvert, adDump and adExport are special cases of features already provided by SQL::Statement and will be re-implemented by using that functionality.
>>>> 
>>>> That all means, future API might be puzzled using roles to avoid strong requirements (Moo or Role::Tiny isn't decided yet) and unfortunately most of existing format-parsers in DarkPAN might require a rewrite. I hope the AnyData resurrection will help to reduce maintaining costs for future and apologize here and now for resulting extra effort when updating.
>>>> 
>>>> Best regards
>>>> -- 
>>>> Jens Rehsack
>>>> pkgsrc, Perl5
>>>> [email protected]
>>>> cpanid: REHSACK
>>>> 
>>>> 
>>>> 
>>>> 
>> 
>> -- 
>> Jens Rehsack
>> [email protected]
>> 
>> 
>> 
>> 
>> 

-- 
Jens Rehsack
[email protected]