Re: Skip rows (number of rows?), column type in library(csv)

On 09/24/2013 10:05 AM, Boris Vassilev wrote:
> Hello,
>
> I am using the (highly useful) library(csv) and I had to do two
> tricks, in order to:
> 1. Skip the header;
> 2. Control (and validate) the datatype of the columns.
>
> It is easiest to explain with the code:
>
> load_csv.pl
> ==
> :- module(load_csv,
>      [load_csv_file/4
>      ]).
>
> :- use_module(library(csv)).
>
> load_csv_file(File, Functor, Skip, ColumnTypes) :-
>      FirstLine is Skip + 1,
>      forall(
>          load_table_row(File, Functor, ColumnTypes, Line, Row),
>          (   Line >= FirstLine
>          ->  assertz(Row)
>          ;   true
>          )
>      ).
>
> load_table_row(File, Functor, ColumnTypes, Line, Row) :-
>      csv_read_file_row(
>          File,
>          R,
>          [convert(false),line(Line)]
>      ),
>      R =.. [row|RawColumns],
>      maplist(convert_field, ColumnTypes, RawColumns, Columns),
>      Row =.. [Functor|Columns].
>
> convert_field(atom, Field, Field).
> convert_field(number, Field, Number) :-
>      atom_number(Field, Number).
> % add additional types as needed
> ==
>
> Is there a way to achieve this using the available options that I didn't see?
>
> Does anyone thinks it would be useful to add options to the library for
> A. Skipping N lines at the beginning;
> B. Loading at most N lines;
> C. Pass a list of datatypes for validation and convertion of the data fields?
>
> In other words, adding options
> A. 'skip(+Integer)' with a default 0;
> B. 'nrows(+Integer)' with a default -1 meaning all (or maybe 'infinite'?)

Yes.  Maybe call them offset(+Skip) and limit(+Count) (using the
common database jargon?)  I'd use =infinite= rather than -1.

> C. 'col_types(+List:atom)' which could be for example 'atom',
> 'number', 'integer', 'float', etc. There is a 'convert(+Boolean)'
> option at the moment but it only gets in the way if the column has
> values like "00", "01", ..., "A0", ..., "QZ", etc.

Yes.  Only, this is calling convert_field/3.  This should properly
call in the calling module (making the CSV predicates meta-predicates).
I'd call the call-back cvs_convert_field/3 and maybe add an option
to specify it.  An alternative might be to call cvs:convert_field/3,
providing a sensible default implementation which first calls a
multifile hook.

> If there is any interest and we agree on the interface I can submit a
> proposed implementation.

If properly implemented and documented, I'm happy to apply the patch.
	
	Thanks --- Jan

>
> Cheers,
> Boris
> _______________________________________________
> SWI-Prolog mailing list
> [email protected]
> https://lists.iai.uni-bonn.de/mailman/listinfo.cgi/swi-prolog
>