Re: Skip rows (number of rows?), column type in library(csv)

Small addition: it the `term` option is provided, the `functor` option, if
provided, must match the name of the term, domain_error otherwise?


On Tue, Oct 29, 2013 at 10:56 AM, Boris Vassilev
<[email protected]>wrote:

> Hello Jan (and others),
>
> finally getting to the suggested additions to library(csv), but having
> looked at the existing functionality and implementation, I need more
> feedback.
>
> If the user wants to provide a term that describes the rows, as the option
> `term`, in the form:
>
> --
> term=row(col1_spec, col2_spec, ..., coln_spec)
> --
> (my original proposal)
>
> then how should relevant existing options be handled?
>
> - `match_arity`
>   * if `match_arity` is false, ignore the `term` option *for the offending
> rows only*
> - `arity`:
>   * if provided, and nonneg, and `match_arity` is true, then it must be
> equal to the number of arguments of the provided term; domain_error
> otherwise?
>   * if a variable, and `match_arity` is true, unify with the arity of all
> rows
>
> Small aside: I know that at the moment I can't say:
>
> --
> csv_read_file(Filename, Rows, [arity(A)]).
> --
>
> but I don't understand why not.
>
> - `convert`
>   * have a `default` column specification that then falls back to the
> action of `convert` (if `convert` is true, try to make a number out of the
> value)?
>
> I have to admit, I did not exactly understand Jan's suggestion how to deal
> with the option `term`. If I understood correctly, add a multi-file
> predicate `csv:convert_field/3` to library(csv) that has definitions for
> `integer`, `float`, `number`, `atom`, and `default` (see above for
> `default`'s meaning):
>
> --
> convert_field(integer, Field, Int) :- ...
> convert_field(float, Field, Float) :- ...
> ...
> --
>
> but allow the user to add clauses to `convert_field/3` in the calling
> module, with the first argument an atom, and use these atoms in the
> description of the `term` option to `csv_read_file/3`.
>
> Thank you for the feedback,
> Boris
> P.S. Someone else would have written the code in the time it took me to
> type in this email.
>
>
> On Mon, Oct 7, 2013 at 4:44 PM, Jan Wielemaker <[email protected]> wrote:
>
>> On 09/24/2013 10:05 AM, Boris Vassilev wrote:
>>
>>> Hello,
>>>
>>> I am using the (highly useful) library(csv) and I had to do two
>>> tricks, in order to:
>>> 1. Skip the header;
>>> 2. Control (and validate) the datatype of the columns.
>>>
>>> It is easiest to explain with the code:
>>>
>>> load_csv.pl
>>> ==
>>> :- module(load_csv,
>>>      [load_csv_file/4
>>>      ]).
>>>
>>> :- use_module(library(csv)).
>>>
>>> load_csv_file(File, Functor, Skip, ColumnTypes) :-
>>>      FirstLine is Skip + 1,
>>>      forall(
>>>          load_table_row(File, Functor, ColumnTypes, Line, Row),
>>>          (   Line >= FirstLine
>>>          ->  assertz(Row)
>>>          ;   true
>>>          )
>>>      ).
>>>
>>> load_table_row(File, Functor, ColumnTypes, Line, Row) :-
>>>      csv_read_file_row(
>>>          File,
>>>          R,
>>>          [convert(false),line(Line)]
>>>      ),
>>>      R =.. [row|RawColumns],
>>>      maplist(convert_field, ColumnTypes, RawColumns, Columns),
>>>      Row =.. [Functor|Columns].
>>>
>>> convert_field(atom, Field, Field).
>>> convert_field(number, Field, Number) :-
>>>      atom_number(Field, Number).
>>> % add additional types as needed
>>> ==
>>>
>>> Is there a way to achieve this using the available options that I didn't
>>> see?
>>>
>>> Does anyone thinks it would be useful to add options to the library for
>>> A. Skipping N lines at the beginning;
>>> B. Loading at most N lines;
>>> C. Pass a list of datatypes for validation and convertion of the data
>>> fields?
>>>
>>> In other words, adding options
>>> A. 'skip(+Integer)' with a default 0;
>>> B. 'nrows(+Integer)' with a default -1 meaning all (or maybe 'infinite'?)
>>>
>>
>> Yes.  Maybe call them offset(+Skip) and limit(+Count) (using the
>> common database jargon?)  I'd use =infinite= rather than -1.
>>
>>
>>  C. 'col_types(+List:atom)' which could be for example 'atom',
>>> 'number', 'integer', 'float', etc. There is a 'convert(+Boolean)'
>>> option at the moment but it only gets in the way if the column has
>>> values like "00", "01", ..., "A0", ..., "QZ", etc.
>>>
>>
>> Yes.  Only, this is calling convert_field/3.  This should properly
>> call in the calling module (making the CSV predicates meta-predicates).
>> I'd call the call-back cvs_convert_field/3 and maybe add an option
>> to specify it.  An alternative might be to call cvs:convert_field/3,
>> providing a sensible default implementation which first calls a
>> multifile hook.
>>
>>
>>  If there is any interest and we agree on the interface I can submit a
>>> proposed implementation.
>>>
>>
>> If properly implemented and documented, I'm happy to apply the patch.
>>
>>         Thanks --- Jan
>>
>>
>>
>>> Cheers,
>>> Boris
>>> ______________________________**_________________
>>> SWI-Prolog mailing list
>>> [email protected].**de <[email protected]>
>>> https://lists.iai.uni-bonn.de/**mailman/listinfo.cgi/swi-**prolog<https://lists.iai.uni-bonn.de/mailman/listinfo.cgi/swi-prolog>
>>>
>>>
>>
>
-------------- next part --------------
HTML attachment scrubbed and removed