Re: DBD::File missing f_ext

On Thu, 18 Sep 2008 10:59:52 -0700, Jeff Zucker <[email protected]>
wrote:

> H.Merijn Brand wrote:
> > Attached patch implements the following:
> >
> >   f_dir => "/data/csv/foo",
> >   f_ext => ".csv",
> >   f_ext => ".csv/i",  # Case ignore on extension, use given on write
> >   f_ext => ".csv/r",  # Extension is required, ignore other files in
> >   f_ext => ".csv/ri", #  f_dir. Options can be combined
>
> I find that overly complicated, doesn't solve much, and could be done 
> easily with a single loop statement in Perl.  The case stuff I would 

The reason I did the /r is that in my case I actually had "aard" and
"aard.csv" and only the latter contained database data

I implemeted /i, because it wasn't too hard to imagine people sending
in zip files with AARD.CSV, and I don't want my script to care.

> find particularly confusing since f_dir and file names (not table names) 
> will need to remain case sensitive on *nix regardless of what you do 
> with the sensitivity of the extension - SQL table names must remain 
> insensitive in all cases so this method of treating a filename as a

That is a good reason to drop the /i support, and make /i default

> tablename would make the entire situation quite confusing.  The reason I 
> say it doesn't solve much is that it only really works if you have a 
> single CSV format (e.g. if you have files that use a semi-colon eol

semi-colon eol's will be the odd-one-out

> you'll need to specify them with $dbh->{csv_tables} attribute anyway.  
> Also it doesn't solve the issue of finding a list of tables - a fuller 
> patch would connect the specifiying of f_dir and f_ext with the 
> $dbh->tables () method.  I guess my main approach is that filenames and 
> tablenames are two different things and that this patch obscures that 

Entirely my POV, which is why I use $h->{f_map} internally to map table
names to file names

> whereas most users will be better off by realizing that they are better 
> off making an explicit association between file names and table names 

Ahhhhrg, no. I have data directories with 400 csv files. You don't
think I would like to specify all of those. f_dir=dta;f_ext=.csv really
made my life very easy there

> with $dbh->{csv_tables}->{file}.  So, let me think about this patch, I'm 
> not ready to decide today.  I will definitely apply your previous 
> patches from rt, thanks for those.

\o/

The reasoning to push this is that we have customers who can easily
dump their data like this, but getting their data in a database format
they use themselves takes ages. Different version of Oracle or OS make
life hard. And analyzing some data from CSV files for a one-shot action
is much much faster that trying to recreate a complete database with
those values. (OK, Postgress goes rather fast, but Oracle is a PITA)

> TIM or others - what do you think?
> 
> Also, thanks for your offer to take over DBD:CSV.  I may sometime take 
> you up on that but am not ready yet, I'd prefer to keep it as long as I 
> have DBD::File, DBD::AnyData, and SQL::Statement since they are all so 
> intimately related.   Is there a way we can be co-maintainers? Can I 
> give you PAUSE uploading rights so that you could directly upload 
> patches we both agree are good changes?
> 
> Thanks for your improvements to the module.

-- 
H.Merijn Brand          Amsterdam Perl Mongers  http://amsterdam.pm.org/
using & porting perl 5.6.2, 5.8.x, 5.10.x, 5.11.x on HP-UX 10.20, 11.00,
11.11, 11.23, and 11.31, SuSE 10.1, 10.2, and 10.3, AIX 5.2, and Cygwin.
http://mirrors.develooper.com/hpux/           http://www.test-smoke.org/
http://qa.perl.org      http://www.goldmark.org/jeff/stupid-disclaimers/