Re: CSV reader: is it possible to disable trim?
Shiro Kawai <[email protected]> Sun, 17 May 2026 20:08:37 -1000
| Newsgroups | gmane.lisp.scheme.gauche |
|---|---|
| Message-ID | <CALN0JNGR3_kWn=Moxdig4vxRKgVM2iQrVxpfLMU7YpxBydckBg@mail.gmail.com> |
--===============1998030849866066848== Content-Type: multipart/alternative; boundary="0000000000005a04420652116589" --0000000000005a04420652116589 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable https://github.com/shirok/Gauche/issues/1271 Decided not to change the default, but made it customizable with a parameter and/or optional argument. Changing the default can cause hard-to-surface bugs in existing code. Let me know if this fix works. --shiro On Sat, May 16, 2026 at 6:47=E2=80=AFPM Shiro Kawai <[email protected]>= wrote: > OK, you convinced me the current implementation is incorrect. I'll > disable it or make it customizable, after pondering the compatibility > implications. > > > > On Sat, May 16, 2026 at 11:11=E2=80=AFAM Jens Thiele <[email protected]> wro= te: > >> Hi, >> >> by default the CSV reader in text.csv trims whitespace. Is it possible >> to disable this? (yes, CSV is a mess). >> >> I didn't know there is a RFC for it. Reading: >> https://www.rfc-editor.org/rfc/rfc4180 >> " >> Within the header and each record, there may be one or more >> fields, separated by commas. Each line should contain the same >> number of fields throughout the file. Spaces are considered part >> of a field and should not be ignored. >> " >> >> which "conflicts" with text.csv: >> "Usually, the whitespaces around the field are ignored." >> >> PS: regarding the CSV mess - quoting the COPY man page from PostgreSQL: >> " >> Note >> >> Many programs produce strange and occasionally perverse CSV files, >> so the file format is more a convention than a standard. Thus you >> might encounter some files that cannot be imported using this >> mechanism, and COPY might produce files that other programs cannot >> process. >> " >> >> $ psql >> karme=3D> select * from csvtest; >> id | name | dummy >> ----+---------------------------------+------- >> 1 | text surrounded by whitespace | 42 >> 2 | whitespace in front | 42 >> 3 | whitespace afterwards | 42 >> >> karme=3D> copy csvtest to stdout with csv; >> 1, text surrounded by whitespace ,42 >> 2, whitespace in front,42 >> 3,whitespace afterwards ,42 >> >> possible workaround: >> >> karme=3D> copy csvtest to stdout with csv force quote name; >> 1," text surrounded by whitespace ",42 >> 2," whitespace in front",42 >> 3,"whitespace afterwards ",42 >> >> >> _______________________________________________ >> Gauche-devel mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/gauche-devel >> > --0000000000005a04420652116589 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_default" style=3D"font-size:large"><a = href=3D"https://github.com/shirok/Gauche/issues/1271">https://github.com/sh= irok/Gauche/issues/1271</a></div><div class=3D"gmail_default" style=3D"font= -size:large"><br></div><div class=3D"gmail_default" style=3D"font-size:larg= e">Decided not to change the default, but=C2=A0made it customizable with a = parameter and/or optional=C2=A0argument.</div><div class=3D"gmail_default" = style=3D"font-size:large">Changing the default can cause hard-to-surface bu= gs in existing code.</div><div class=3D"gmail_default" style=3D"font-size:l= arge">Let me know=C2=A0if this fix works.</div><div class=3D"gmail_default"= style=3D"font-size:large"><br></div><div class=3D"gmail_default" style=3D"= font-size:large">--shiro</div><div class=3D"gmail_default" style=3D"font-si= ze:large"><br></div><div class=3D"gmail_default" style=3D"font-size:large">= <br></div></div><br><div class=3D"gmail_quote gmail_quote_container"><div d= ir=3D"ltr" class=3D"gmail_attr">On Sat, May 16, 2026 at 6:47=E2=80=AFPM Shi= ro Kawai <<a href=3D"mailto:[email protected]">[email protected]= </a>> wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:= 0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">= <div dir=3D"ltr"><div class=3D"gmail_default" style=3D"font-size:large">OK,= you convinced me the current implementation=C2=A0is incorrect.=C2=A0 =C2= =A0I'll disable=C2=A0it or make it customizable, after pondering the co= mpatibility implications.</div><div class=3D"gmail_default" style=3D"font-s= ize:large"><br></div><div class=3D"gmail_default" style=3D"font-size:large"= ><br></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"g= mail_attr">On Sat, May 16, 2026 at 11:11=E2=80=AFAM Jens Thiele <<a href= =3D"mailto:[email protected]" target=3D"_blank">[email protected]</a>> wrote:<= br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8e= x;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi,<br> <br> by default the CSV reader in text.csv trims whitespace. Is it possible<br> to disable this? (yes, CSV is a mess).<br> <br> I didn't know there is a RFC for it. Reading:<br> <a href=3D"https://www.rfc-editor.org/rfc/rfc4180" rel=3D"noreferrer" targe= t=3D"_blank">https://www.rfc-editor.org/rfc/rfc4180</a><br> "<br> Within the header and each record, there may be one or more<br> fields, separated by commas.=C2=A0 Each line should contain the same<br> number of fields throughout the file.=C2=A0 Spaces are considered part<br> of a field and should not be ignored.<br> "<br> <br> which "conflicts" with text.csv:<br> "Usually, the whitespaces around the field are ignored."<br> <br> PS: regarding the CSV mess - quoting the COPY man page from PostgreSQL:<br> "<br> Note<br> <br> Many programs produce strange and occasionally perverse CSV files,<br> so the file format is more a convention than a standard. Thus you<br> might encounter some files that cannot be imported using this<br> mechanism, and COPY might produce files that other programs cannot<br> process.<br> "<br> <br> $ psql<br> karme=3D> select * from csvtest;<br> =C2=A0id |=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 name=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0| dummy <br> ----+---------------------------------+-------<br> =C2=A0 1 |=C2=A0 text surrounded by whitespace=C2=A0 |=C2=A0 =C2=A0 42<br> =C2=A0 2 |=C2=A0 whitespace in front=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 |=C2=A0 =C2=A0 42<br> =C2=A0 3 | whitespace afterwards=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0|= =C2=A0 =C2=A0 42<br> <br> karme=3D> copy csvtest to stdout with csv;<br> 1, text surrounded by whitespace ,42<br> 2, whitespace in front,42<br> 3,whitespace afterwards ,42<br> <br> possible workaround:<br> <br> karme=3D> copy csvtest to stdout with csv force quote name;<br> 1," text surrounded by whitespace ",42<br> 2," whitespace in front",42<br> 3,"whitespace afterwards ",42<br> <br> <br> _______________________________________________<br> Gauche-devel mailing list<br> <a href=3D"mailto:[email protected]" target=3D"_blank">Gau= [email protected]</a><br> <a href=3D"https://lists.sourceforge.net/lists/listinfo/gauche-devel" rel= =3D"noreferrer" target=3D"_blank">https://lists.sourceforge.net/lists/listi= nfo/gauche-devel</a><br> </blockquote></div> </blockquote></div> --0000000000005a04420652116589-- --===============1998030849866066848== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline --===============1998030849866066848== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Gauche-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/gauche-devel --===============1998030849866066848==--