Re: BUG #16200: returned data from ESQL/C FETCH is trampling outside assigned memory for CHAR column

--Nq2Wo0NMKNjxTN9z
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

El d=C3=ADa viernes, enero 10, 2020 a las 01:43:38p. m. +0100, Michael Mesk=
es escribi=C3=B3:

> Hi,
>=20
> > We encounter the following problem with ESQL/C: Imagine a table with=20
>=20
> I assume you mean ECPG, right? ESQL/C would be the Informix compiler.

Hi.

I mean with ESQL/C the general term "Embedded SQL in C". Of course I do
know that for PostgreSQL the precompiler is named "ecpg".

> > In the database the CHAR column can contain not only 16 bytes, but 16
> > Unicode chars,
> > which are longer than 16 bytes if one or more of the chars is an UTF-
> > 8
> > multibyte
> > encoded char.
> > ...
> > the DATE. Now the function ECPGdo() places the DATE as "MM.DD.YYYY"
> > into the area pointed to for the 2nd argument and with this
> > overwrites
> > the NULL terminator of the string[17] element. Result is later a
> > SIGSEGV because the expected string in string[17] is not NULL
> > terminated anymore :-)
> >=20
> > I would call it a bug, that ECPGdo() puts more than 17 bytes (16
> > bytes +
> > NULL) as return into the place pointed to by the host var pointer
> > when
> > the column in the database has more (UTF-8) chars as will fit into
> > 16+1 byte.
>=20
> Actually I am not sure if this is a bug. I do not remember the standard
> asking for a null termination at the end of a partial string copy.

Neither do I know if there is some written standard, but char strings IMHO
should be NULL terminated. Our application expects in this example no
more than 16 bytes and that's why we provide a host variable as the
struct member of 17 bytes so the NULL fits into.

I digged into the sources of the ecpglib and this small fix solved our prob=
lem:

postgresql-11.4/src/interfaces/ecpg> diff ecpglib/data.c ecpglib/data.c.ori=
g         =20
527,531c527
<                                                       /* [email protected]=
: strncpy() only varcharsize-1 */
<                                                       strncpy(str, pval, =
varcharsize-1);
<                                                       /* [email protected]=
: and terminate the string */
<                                                       str[varcharsize - 1=
] =3D '\0';
<                                                       ecpg_log("DEBUG ESQ=
L/C: result [%s] len %d\n", str, strlen(str));
---
>                                                       strncpy(str, pval, =
varcharsize);

If I now store 16 UTF-8 chars German Umlaut '=C3=A4' into the column, the E=
CPG log is this:

[30481] [10.01.2020 12:29:28:936]: ecpg_get_data on line 3181: RESULT: =C3=
=A4=C3=A4=C3=A4=C3=A4=C3=A4=C3=A4=C3=A4=C3=A4=C3=A4=C3=A4=C3=A4=C3=A4=C3=A4=
=C3=A4=C3=A4=C3=A4 offset: 1600; array: no
[30481] [10.01.2020 12:29:28:936]: DEBUG ESQL/C: varcharsize 17 size 32
[30481] [10.01.2020 12:29:28:936]: DEBUG ESQL/C: result [=C3=A4=C3=A4=C3=A4=
=C3=A4=C3=A4=C3=A4=C3=A4=C3=A4] len 16

(The two lines with DEBUG are added by me in execute.c to understand
better the problem).

ECPG would return as RESULT as it says '=C3=A4=C3=A4=C3=A4=C3=A4=C3=A4=C3=
=A4=C3=A4=C3=A4=C3=A4=C3=A4=C3=A4=C3=A4=C3=A4=C3=A4=C3=A4=C3=A4' which is 32
bytes (16 UTF-8 chars), our varcharsize is only 17 and we now truncate the =
RESULT=20
to 16 bytes (8 UTF-8 '=C3=A4') and a trailing \0 with my proposed change an=
d all is fine.

Btw: truncating UTF-8 strings in any case brings the risk, that the resulti=
ng
string is not longer clean UTF-8 if you hit into the middle of a multibyte
char. But such issue should be handled by the application and not by ECPG.


> Please correct me if I am wrong. What it does ask for is setting the
> indicator accordingly. However, you do not mention any indicator, so I
> wonder if you checked that one at all. If the string is truncated and
> the appropriate error action is not taken, that would definitely
> qualify as a bug.
>=20
> Could you please verify if the indicator is set accordingly?=20

If I read the docs here https://www.postgresql.org/docs/11/ecpg-variables.h=
tml#ECPG-INDICATORS
indicator vars are meant to show null values in the table and not the
string truncation. Am I wrong? We do not use indicators at all, but
catch the error condition -213.

In any case, thanks for your attention.

	matthias

--=20
Matthias Apitz, =E2=9C=89 [email protected], http://www.unixarea.de/ +49-176=
-38902045
Public GnuPG key: http://www.unixarea.de/key.pub

"Glaube wenig, hinterfrage alles, denke selbst: Wie man Manipulationen durc=
hschaut"
"Believe little, scrutinise all, think by your own: How see through manipul=
ations"
ISBN-10: 386489218X

--Nq2Wo0NMKNjxTN9z
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEXmn7rBYYViyzy/vBR8z35Hb+nREFAl4YkbIACgkQR8z35Hb+
nRElXQ/8DccX5FDR1w/6Z1a1zLTiQtRfx+c1LHpiSczg/AcMr84LKsqwvuIsyiHq
BTTdIQj6g3gOkJptX+mllDb7mLzS6x6c3Bc8kZA1CVS6MdjRQiHMJQL+MxeIHejS
hFbTSx1XTNSxQB8OikmTF84AE/uNpXVcVUtP/oewbZn883o40f7wIDt4zk2OFt+m
NWtQ29pdFL4I6uUYgd3tWZ6QPGbpvMI3xCjaBOoOfCjVEc6DYlrL5bdGXVwcWWVo
c1cDBmrSwam2SacHvUO1UA08PLy8jU3KXdFwRfLA4T1jejQVvyM9GbVUfjpk7VrD
I//c06A24vgFmeIEUzzic8eho23XjSRwVdGZUgpXDc5N/J3CcOD1GSvb0Wpp36z3
VmN/t4Ftl+PsogmOdAvq78e782gLRq4xNGsTwq5+yt1+8NUQXuiJVOosyNIzr+Ht
eUqsiytkM4rGVhE+PFOMdkGfFLcSBJ+FWEQCwG+FdECgQSZbg6AVFCiGDUtJem3E
sC23V3Mffol/4kYbwgwG+P1ZvV2TP77/Cc/3Drc8Huqnf4O17RqPd7sbiiQvuK0O
58jabXuwG93lPgMRh3Opg7rsinM2vv5nKvFqk+3VchGmvR8xAzuKOjltaP8utA7N
I3lkcZ2Yu76lDHhhZ+8c1WUo/T4tcD7LJ1b+s0rgLZt1sLYSiv0=
=fy9Z
-----END PGP SIGNATURE-----

--Nq2Wo0NMKNjxTN9z--