Re: Dialyzer problems with zlib 1.2.10 and 1.2.11
Sverker Eriksson <[email protected]> Fri, 20 Jan 2017 17:45:52 +0100
| Newsgroups | gmane.comp.lang.erlang.bugs |
|---|---|
| Organization | Ericsson AB |
| Message-ID | <[email protected]> |
--===============1399796790841753914==
Content-Type: multipart/alternative;
boundary="------------38A20098A30B427A13973CE8"
--------------38A20098A30B427A13973CE8
Content-Type: text/plain; charset="windows-1252"; format=flowed
Content-Transfer-Encoding: quoted-printable
Correction: Bug exists since OTP-17.0.
(and i tags R16B02_yielding_binary_to_term and=20
OTP_R16B03_yielding_binary_to_term)
/Sverker
On 01/20/2017 05:15 PM, Sverker Eriksson wrote:
> This is indeed a problem in Erlang VM code (shallow copy of inflate=20
> state)
> that has existed since R16B03, but not caused actual problem until=20
> zlib v1.2.9.
>
> Fix coming up. Here is a preliminary patch for the impatient.
>
> diff --git a/erts/emulator/beam/external.c=20
> b/erts/emulator/beam/external.c
> index beed847..1c4fff5 100644
> --- a/erts/emulator/beam/external.c
> +++ b/erts/emulator/beam/external.c
> @@ -1431,6 +1431,10 @@ static B2TContext* b2t_export_context(Process*=20
> p, B2TContext* src)
> if (ctx->state >=3D B2TDecode && ctx->u.dc.next =3D=3D &src->u.dc.=
res) {
> ctx->u.dc.next =3D &ctx->u.dc.res;
> }
> + else if (ctx->state =3D=3D B2TUncompressChunk) {
> + int cres =3D inflateCopy(&ctx->u.uc.stream, &src->u.uc.stream)=
;
> + ASSERT(cres =3D=3D Z_OK); (void)cres;
> + }
> hp =3D HAlloc(p, PROC_BIN_SIZE);
> ctx->trap_bin =3D erts_mk_magic_binary_term(&hp, &MSO(p), context_=
b);
> return ctx;
>
>
> /Sverker, Erlang/OTP
>
>
> On 01/20/2017 02:49 AM, Jeremy Huffman wrote:
>> I opened a Github issue with zlib.=20
>> https://github.com/madler/zlib/issues/206.
>> Mark Adler (zlib maintainer's) response:
>>
>> "Isolating it to that commit points to a problem in the application=20
>> code,
>> where it must be inadvertently stomping on the deflate state, e.g.=20
>> with an
>> out-of-bounds write into memory, or perhaps that the code is trying=20
>> to use
>> the deflate state after it has been closed. The only change that commi=
t
>> made was to check the integrity of the deflate structure more=20
>> thoroughly on
>> each call of a deflate* function."
>>
>> On Thu, Jan 19, 2017 at 2:11 PM, Michel Boaventura <
>> [email protected]> wrote:
>>
>>> Hi,
>>>
>>> I've done the bisect and find the culprit: https://github.com/
>>> madler/zlib/commit/b516b4bdd7c0c9f0858adfebf732089014f7b282. Before=20
>>> this
>>> commit term_to_binary works and stop doing so afterwards. I will have=
a
>>> look at the changes and see if I can figure out what happened.
>>>
>>> Cheers,
>>>
>>>
>>> On 19 January 2017 at 16:15, Michel Boaventura <
>>> [email protected]> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I'm indeed using zlib 1.2.11 on my gentoo. I can't downgrade it, sin=
ce
>>>> all the other versions were removed from portage.
>>>>
>>>> I will clone zlib repo and see if I can bisect the problem.
>>>>
>>>> Thanks!
>>>>
>>>> On 19 January 2017 at 15:45, Jeremy Huffman <[email protected]=
m>
>>>> wrote:
>>>>
>>>>> Yes it's exactly the same error message from dialyzer. And the=20
>>>>> fact that
>>>>> he's getting it on Gentoo which builds from source suggests that=20
>>>>> it is not
>>>>> simply a matter of recompiling the dependency chain, which was a=20
>>>>> suggestion
>>>>> in the Arch board. There was another app in Arch that also had a=20
>>>>> problem
>>>>> pinned on zlib 1.2.11.
>>>>>
>>>>>
>>>>> On Thu, Jan 19, 2017 at 11:33 AM Kostis Sagonas <[email protected]>=
>>>>> wrote:
>>>>>
>>>>>> On 01/19/2017 03:42 AM, Jeremy Huffman wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>> I'm an Arch Linux user and picked up an update a few days ago tha=
t
>>>>>> broke
>>>>>>
>>>>>>> dialyzer. I bisected the last few days of updates and then narrow=
ed
>>>>>> the
>>>>>>
>>>>>>> problem to zlib 1.2.10, which was released January 2nd. 1.2.11 wa=
s
>>>>>>> released on the 15th as an emergency bug fix and does not fix the=
>>>>>>> problem. Reverting my system back to 1.2.8 (the previous version
>>>>>>> packaged for Arch) did resolve the issue.
>>>>>>> It seems doubtful this is an Erlang problem, but I doubt I'm=20
>>>>>>> going to
>>>>>>> write a test program to demonstrate the problem to them. I=20
>>>>>>> thought I
>>>>>>> should at least report the issue in case others encounter it.
>>>>>>> To reproduce, one would need only install zlib 1.2.10 and then ru=
n:
>>>>>>> dialyzer --verbose --build_plt --apps erts --output_plt test.plt
>>>>>>> Output would be along the lines of:
>>>>>>> dialyzer: Could not get abstract code for file:
>>>>>>> /usr/lib/erlang/lib/erts-8.2/ebin/erlang.beam (please recompile i=
t
>>>>>> with
>>>>>>
>>>>>>> +debug_info)
>>>>>>> There are also errors when simply trying to do success typing=20
>>>>>>> analysis
>>>>>>> *using* any pre-existing PLT file, along lines of "this isn't a P=
LT
>>>>>>> file". The errors are not dependent upon the version of Erlang
>>>>>> installed
>>>>>>
>>>>>>> - at least anything I tried that was released on Arch in the 19.x=
>>>>>> branch
>>>>>>
>>>>>>> will reproduce the problem.
>>>>>>> Anyway, I hope this report helps someone and I would be curious i=
f
>>>>>>> anyone else reproduces it, or especially if they fail to=20
>>>>>>> reproduce it.
>>>>>>
>>>>>>
>>>>>> Earlier today (yesterday?), there was the following question on th=
e
>>>>>>
>>>>>> erlang-questions mailing list:
>>>>>>
>>>>>>
>>>>>>
>>>>>> http://erlang.org/pipermail/erlang-questions/2017-January/0
>>>>>> 91434.html
>>>>>>
>>>>>>
>>>>>>
>>>>>> I am willing to bet that problem with binary_to_term is also=20
>>>>>> caused by
>>>>>>
>>>>>> zlib troubles.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Perhaps Michel (cc:) can inform us about his zlib version.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Kostis
>>>>>>
>>>>>>
>>>>
>>>> --=20
>>>> Michel Almada de Castro Boaventura
>>>> Analista de Sistemas
>>>> Laborat=F3rio de Software Livre - LSL
>>>>
>>>
>>>
>>> --=20
>>> Michel Almada de Castro Boaventura
>>> Analista de Sistemas
>>> Laborat=F3rio de Software Livre - LSL
>>>
>>
>>
>> _______________________________________________
>> erlang-bugs mailing list
>> [email protected]
>> http://erlang.org/mailman/listinfo/erlang-bugs
>
>
>
>
> _______________________________________________
> erlang-bugs mailing list
> [email protected]
> http://erlang.org/mailman/listinfo/erlang-bugs
--------------38A20098A30B427A13973CE8
Content-Type: text/html; charset="windows-1252"
Content-Transfer-Encoding: quoted-printable
<html>
<head>
<meta content=3D"text/html; charset=3Dwindows-1252"
http-equiv=3D"Content-Type">
</head>
<body text=3D"#000000" bgcolor=3D"#FFFFFF">
<p>Correction: Bug exists since OTP-17.0.
<br>
<br>
(and i tags R16B02_yielding_binary_to_term and
OTP_R16B03_yielding_binary_to_term)
<br>
<br>
/Sverker
</p>
<br>
<div class=3D"moz-cite-prefix">On 01/20/2017 05:15 PM, Sverker
Eriksson wrote:<br>
</div>
<blockquote
cite=3D"mid:[email protected]"
type=3D"cite">This is indeed a problem in Erlang VM code (shallow
copy of inflate state)
<br>
that has existed since R16B03, but not caused actual problem until
zlib v1.2.9.
<br>
<br>
Fix coming up. Here is a preliminary patch for the impatient.
<br>
<br>
diff --git a/erts/emulator/beam/external.c
b/erts/emulator/beam/external.c
<br>
index beed847..1c4fff5 100644
<br>
--- a/erts/emulator/beam/external.c
<br>
+++ b/erts/emulator/beam/external.c
<br>
@@ -1431,6 +1431,10 @@ static B2TContext*
b2t_export_context(Process* p, B2TContext* src)
<br>
=A0=A0=A0=A0 if (ctx->state >=3D B2TDecode &&
ctx->u.dc.next =3D=3D &src->u.dc.res) {
<br>
=A0=A0=A0=A0=A0=A0=A0=A0 ctx->u.dc.next =3D &ctx->u.dc.re=
s;
<br>
=A0=A0=A0=A0 }
<br>
+=A0=A0=A0 else if (ctx->state =3D=3D B2TUncompressChunk) {
<br>
+=A0=A0=A0=A0=A0=A0=A0 int cres =3D inflateCopy(&ctx->u.uc.s=
tream,
&src->u.uc.stream);
<br>
+=A0=A0=A0=A0=A0=A0=A0 ASSERT(cres =3D=3D Z_OK); (void)cres;
<br>
+=A0=A0=A0 }
<br>
=A0=A0=A0=A0 hp =3D HAlloc(p, PROC_BIN_SIZE);
<br>
=A0=A0=A0=A0 ctx->trap_bin =3D erts_mk_magic_binary_term(&hp=
,
&MSO(p), context_b);
<br>
=A0=A0=A0=A0 return ctx;
<br>
<br>
<br>
/Sverker, Erlang/OTP
<br>
<br>
<br>
On 01/20/2017 02:49 AM, Jeremy Huffman wrote:
<br>
<blockquote type=3D"cite">I opened a Github issue with zlib.
<a class=3D"moz-txt-link-freetext" href=3D"https://github.com/mad=
ler/zlib/issues/206">https://github.com/madler/zlib/issues/206</a>.
<br>
Mark Adler (zlib maintainer's) response:
<br>
<br>
"Isolating it to that commit points to a problem in the
application code,
<br>
where it must be inadvertently stomping on the deflate state,
e.g. with an
<br>
out-of-bounds write into memory, or perhaps that the code is
trying to use
<br>
the deflate state after it has been closed. The only change that
commit
<br>
made was to check the integrity of the deflate structure more
thoroughly on
<br>
each call of a deflate* function."
<br>
<br>
On Thu, Jan 19, 2017 at 2:11 PM, Michel Boaventura <
<br>
<a class=3D"moz-txt-link-abbreviated" href=3D"mailto:michel.boave=
[email protected]">[email protected]</a>> wrote:
<br>
<br>
<blockquote type=3D"cite">Hi,
<br>
<br>
I've done the bisect and find the culprit: <a class=3D"moz-txt-=
link-freetext" href=3D"https://github.com/">https://github.com/</a>
<br>
madler/zlib/commit/b516b4bdd7c0c9f0858adfebf732089014f7b282.
Before this
<br>
commit term_to_binary works and stop doing so afterwards. I
will have a
<br>
look at the changes and see if I can figure out what happened.
<br>
<br>
Cheers,
<br>
<br>
<br>
On 19 January 2017 at 16:15, Michel Boaventura <
<br>
<a class=3D"moz-txt-link-abbreviated" href=3D"mailto:michel.boa=
[email protected]">[email protected]</a>> wrote:
<br>
<br>
<blockquote type=3D"cite">Hi all,
<br>
<br>
I'm indeed using zlib 1.2.11 on my gentoo. I can't downgrade
it, since
<br>
all the other versions were removed from portage.
<br>
<br>
I will clone zlib repo and see if I can bisect the problem.
<br>
<br>
Thanks!
<br>
<br>
On 19 January 2017 at 15:45, Jeremy Huffman
<a class=3D"moz-txt-link-rfc2396E" href=3D"mailto:jeremy@jere=
myhuffman.com"><[email protected]></a>
<br>
wrote:
<br>
<br>
<blockquote type=3D"cite">Yes it's exactly the same error
message from dialyzer. And the fact that
<br>
he's getting it on Gentoo which builds from source
suggests that it is not
<br>
simply a matter of recompiling the dependency chain, which
was a suggestion
<br>
in the Arch board. There was another app in Arch that also
had a problem
<br>
pinned on zlib 1.2.11.
<br>
<br>
<br>
On Thu, Jan 19, 2017 at 11:33 AM Kostis Sagonas
<a class=3D"moz-txt-link-rfc2396E" href=3D"mailto:kostis@cs=
=2Entua.gr"><[email protected]></a>
<br>
wrote:
<br>
<br>
<blockquote type=3D"cite">On 01/19/2017 03:42 AM, Jeremy
Huffman wrote:
<br>
<br>
<blockquote type=3D"cite">Hi,
<br>
I'm an Arch Linux user and picked up an update a few
days ago that
<br>
</blockquote>
broke
<br>
<br>
<blockquote type=3D"cite">dialyzer. I bisected the last
few days of updates and then narrowed
<br>
</blockquote>
the
<br>
<br>
<blockquote type=3D"cite">problem to zlib 1.2.10, which
was released January 2nd. 1.2.11 was
<br>
released on the 15th as an emergency bug fix and does
not fix the
<br>
problem. Reverting my system back to 1.2.8 (the
previous version
<br>
packaged for Arch) did resolve the issue.
<br>
It seems doubtful this is an Erlang problem, but I
doubt I'm going to
<br>
write a test program to demonstrate the problem to
them.=A0 I thought I
<br>
should at least report the issue in case others
encounter it.
<br>
To reproduce, one would need only install zlib 1.2.10
and then run:
<br>
dialyzer --verbose --build_plt --apps erts
--output_plt test.plt
<br>
Output would be along the lines of:
<br>
dialyzer: Could not get abstract code for file:
<br>
/usr/lib/erlang/lib/erts-8.2/ebin/erlang.beam (please
recompile it
<br>
</blockquote>
with
<br>
<br>
<blockquote type=3D"cite">+debug_info)
<br>
There are also errors when simply trying to do success
typing analysis
<br>
*using* any pre-existing PLT file, along lines of
"this isn't a PLT
<br>
file". The errors are not dependent upon the version
of Erlang
<br>
</blockquote>
installed
<br>
<br>
<blockquote type=3D"cite">- at least anything I tried tha=
t
was released on Arch in the 19.x
<br>
</blockquote>
branch
<br>
<br>
<blockquote type=3D"cite">will reproduce the problem.
<br>
Anyway, I hope this report helps someone and I would
be curious if
<br>
anyone else reproduces it, or especially if they fail
to reproduce it.
<br>
</blockquote>
<br>
<br>
Earlier today (yesterday?), there was the following
question on the
<br>
<br>
erlang-questions mailing list:
<br>
<br>
<br>
<br>
=A0=A0=A0
<a class=3D"moz-txt-link-freetext" href=3D"http://erlang.=
org/pipermail/erlang-questions/2017-January/0">http://erlang.org/pipermai=
l/erlang-questions/2017-January/0</a>
<br>
91434.html
<br>
<br>
<br>
<br>
I am willing to bet that problem with binary_to_term is
also caused by
<br>
<br>
zlib troubles.
<br>
<br>
<br>
<br>
Perhaps Michel (cc:) can inform us about his zlib
version.
<br>
<br>
<br>
<br>
Kostis
<br>
<br>
<br>
</blockquote>
</blockquote>
<br>
--
<br>
Michel Almada de Castro Boaventura
<br>
Analista de Sistemas
<br>
Laborat=F3rio de Software Livre - LSL
<br>
<br>
</blockquote>
<br>
<br>
--
<br>
Michel Almada de Castro Boaventura
<br>
Analista de Sistemas
<br>
Laborat=F3rio de Software Livre - LSL
<br>
<br>
</blockquote>
<br>
<br>
_______________________________________________
<br>
erlang-bugs mailing list
<br>
<a class=3D"moz-txt-link-abbreviated" href=3D"mailto:erlang-bugs@=
erlang.org">[email protected]</a>
<br>
<a class=3D"moz-txt-link-freetext" href=3D"http://erlang.org/mail=
man/listinfo/erlang-bugs">http://erlang.org/mailman/listinfo/erlang-bugs<=
/a>
<br>
</blockquote>
<br>
<br>
<br>
<fieldset class=3D"mimeAttachmentHeader"></fieldset>
<br>
<pre wrap=3D"">_______________________________________________
erlang-bugs mailing list
<a class=3D"moz-txt-link-abbreviated" href=3D"mailto:[email protected]=
rg">[email protected]</a>
<a class=3D"moz-txt-link-freetext" href=3D"http://erlang.org/mailman/list=
info/erlang-bugs">http://erlang.org/mailman/listinfo/erlang-bugs</a>
</pre>
</blockquote>
<br>
</body>
</html>
--------------38A20098A30B427A13973CE8--
--===============1399796790841753914==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
erlang-bugs mailing list
[email protected]
http://erlang.org/mailman/listinfo/erlang-bugs
--===============1399796790841753914==--