Re: Skip rows (number of rows?), column type in library(csv)

Hello Jan (and others),

finally getting to the suggested additions to library(csv), but having
looked at the existing functionality and implementation, I need more
feedback.

If the user wants to provide a term that describes the rows, as the option
`term`, in the form:

--
term=row(col1_spec, col2_spec, ..., coln_spec)
--
(my original proposal)

then how should relevant existing options be handled?

- `match_arity`
  * if `match_arity` is false, ignore the `term` option *for the offending
rows only*
- `arity`:
  * if provided, and nonneg, and `match_arity` is true, then it must be
equal to the number of arguments of the provided term; domain_error
otherwise?
  * if a variable, and `match_arity` is true, unify with the arity of all
rows

Small aside: I know that at the moment I can't say:

--
csv_read_file(Filename, Rows, [arity(A)]).
--

but I don't understand why not.

- `convert`
  * have a `default` column specification that then falls back to the
action of `convert` (if `convert` is true, try to make a number out of the
value)?

I have to admit, I did not exactly understand Jan's suggestion how to deal
with the option `term`. If I understood correctly, add a multi-file
predicate `csv:convert_field/3` to library(csv) that has definitions for
`integer`, `float`, `number`, `atom`, and `default` (see above for
`default`'s meaning):

--
convert_field(integer, Field, Int) :- ...
convert_field(float, Field, Float) :- ...
...
--

but allow the user to add clauses to `convert_field/3` in the calling
module, with the first argument an atom, and use these atoms in the
description of the `term` option to `csv_read_file/3`.

Thank you for the feedback,
Boris
P.S. Someone else would have written the code in the time it took me to
type in this email.


On Mon, Oct 7, 2013 at 4:44 PM, Jan Wielemaker <[email protected]> wrote:

> On 09/24/2013 10:05 AM, Boris Vassilev wrote:
>
>> Hello,
>>
>> I am using the (highly useful) library(csv) and I had to do two
>> tricks, in order to:
>> 1. Skip the header;
>> 2. Control (and validate) the datatype of the columns.
>>
>> It is easiest to explain with the code:
>>
>> load_csv.pl
>> ==
>> :- module(load_csv,
>>      [load_csv_file/4
>>      ]).
>>
>> :- use_module(library(csv)).
>>
>> load_csv_file(File, Functor, Skip, ColumnTypes) :-
>>      FirstLine is Skip + 1,
>>      forall(
>>          load_table_row(File, Functor, ColumnTypes, Line, Row),
>>          (   Line >= FirstLine
>>          ->  assertz(Row)
>>          ;   true
>>          )
>>      ).
>>
>> load_table_row(File, Functor, ColumnTypes, Line, Row) :-
>>      csv_read_file_row(
>>          File,
>>          R,
>>          [convert(false),line(Line)]
>>      ),
>>      R =.. [row|RawColumns],
>>      maplist(convert_field, ColumnTypes, RawColumns, Columns),
>>      Row =.. [Functor|Columns].
>>
>> convert_field(atom, Field, Field).
>> convert_field(number, Field, Number) :-
>>      atom_number(Field, Number).
>> % add additional types as needed
>> ==
>>
>> Is there a way to achieve this using the available options that I didn't
>> see?
>>
>> Does anyone thinks it would be useful to add options to the library for
>> A. Skipping N lines at the beginning;
>> B. Loading at most N lines;
>> C. Pass a list of datatypes for validation and convertion of the data
>> fields?
>>
>> In other words, adding options
>> A. 'skip(+Integer)' with a default 0;
>> B. 'nrows(+Integer)' with a default -1 meaning all (or maybe 'infinite'?)
>>
>
> Yes.  Maybe call them offset(+Skip) and limit(+Count) (using the
> common database jargon?)  I'd use =infinite= rather than -1.
>
>
>  C. 'col_types(+List:atom)' which could be for example 'atom',
>> 'number', 'integer', 'float', etc. There is a 'convert(+Boolean)'
>> option at the moment but it only gets in the way if the column has
>> values like "00", "01", ..., "A0", ..., "QZ", etc.
>>
>
> Yes.  Only, this is calling convert_field/3.  This should properly
> call in the calling module (making the CSV predicates meta-predicates).
> I'd call the call-back cvs_convert_field/3 and maybe add an option
> to specify it.  An alternative might be to call cvs:convert_field/3,
> providing a sensible default implementation which first calls a
> multifile hook.
>
>
>  If there is any interest and we agree on the interface I can submit a
>> proposed implementation.
>>
>
> If properly implemented and documented, I'm happy to apply the patch.
>
>         Thanks --- Jan
>
>
>
>> Cheers,
>> Boris
>> ______________________________**_________________
>> SWI-Prolog mailing list
>> [email protected].**de <[email protected]>
>> https://lists.iai.uni-bonn.de/**mailman/listinfo.cgi/swi-**prolog<https://lists.iai.uni-bonn.de/mailman/listinfo.cgi/swi-prolog>
>>
>>
>
-------------- next part --------------
HTML attachment scrubbed and removed