Re: Dialyzer problems with zlib 1.2.10 and 1.2.11
Sverker Eriksson <[email protected]> Fri, 20 Jan 2017 17:15:38 +0100
| Newsgroups | gmane.comp.lang.erlang.bugs |
|---|---|
| Organization | Ericsson AB |
| Message-ID | <[email protected]> |
--===============0543584806586181690==
Content-Type: multipart/alternative;
boundary="------------DB52A237CC90BD817F0E4DA8"
--------------DB52A237CC90BD817F0E4DA8
Content-Type: text/plain; charset="windows-1252"; format=flowed
Content-Transfer-Encoding: quoted-printable
This is indeed a problem in Erlang VM code (shallow copy of inflate state=
)
that has existed since R16B03, but not caused actual problem until zlib=20
v1.2.9.
Fix coming up. Here is a preliminary patch for the impatient.
diff --git a/erts/emulator/beam/external.c b/erts/emulator/beam/external.=
c
index beed847..1c4fff5 100644
--- a/erts/emulator/beam/external.c
+++ b/erts/emulator/beam/external.c
@@ -1431,6 +1431,10 @@ static B2TContext* b2t_export_context(Process* p, =
B2TContext* src)
if (ctx->state >=3D B2TDecode && ctx->u.dc.next =3D=3D &src->u.dc.r=
es) {
ctx->u.dc.next =3D &ctx->u.dc.res;
}
+ else if (ctx->state =3D=3D B2TUncompressChunk) {
+ int cres =3D inflateCopy(&ctx->u.uc.stream, &src->u.uc.stream);
+ ASSERT(cres =3D=3D Z_OK); (void)cres;
+ }
hp =3D HAlloc(p, PROC_BIN_SIZE);
ctx->trap_bin =3D erts_mk_magic_binary_term(&hp, &MSO(p), context_b=
);
return ctx;
/Sverker, Erlang/OTP
On 01/20/2017 02:49 AM, Jeremy Huffman wrote:
> I opened a Github issue with zlib. https://github.com/madler/zlib/issue=
s/206.
> Mark Adler (zlib maintainer's) response:
>
> "Isolating it to that commit points to a problem in the application cod=
e,
> where it must be inadvertently stomping on the deflate state, e.g. with=
an
> out-of-bounds write into memory, or perhaps that the code is trying to =
use
> the deflate state after it has been closed. The only change that commit=
> made was to check the integrity of the deflate structure more thoroughl=
y on
> each call of a deflate* function."
>
> On Thu, Jan 19, 2017 at 2:11 PM, Michel Boaventura <
> [email protected]> wrote:
>
>> Hi,
>>
>> I've done the bisect and find the culprit: https://github.com/
>> madler/zlib/commit/b516b4bdd7c0c9f0858adfebf732089014f7b282. Before th=
is
>> commit term_to_binary works and stop doing so afterwards. I will have =
a
>> look at the changes and see if I can figure out what happened.
>>
>> Cheers,
>>
>>
>> On 19 January 2017 at 16:15, Michel Boaventura <
>> [email protected]> wrote:
>>
>>> Hi all,
>>>
>>> I'm indeed using zlib 1.2.11 on my gentoo. I can't downgrade it, sinc=
e
>>> all the other versions were removed from portage.
>>>
>>> I will clone zlib repo and see if I can bisect the problem.
>>>
>>> Thanks!
>>>
>>> On 19 January 2017 at 15:45, Jeremy Huffman <[email protected]=
>
>>> wrote:
>>>
>>>> Yes it's exactly the same error message from dialyzer. And the fact =
that
>>>> he's getting it on Gentoo which builds from source suggests that it =
is not
>>>> simply a matter of recompiling the dependency chain, which was a sug=
gestion
>>>> in the Arch board. There was another app in Arch that also had a pro=
blem
>>>> pinned on zlib 1.2.11.
>>>>
>>>>
>>>> On Thu, Jan 19, 2017 at 11:33 AM Kostis Sagonas <[email protected]>
>>>> wrote:
>>>>
>>>>> On 01/19/2017 03:42 AM, Jeremy Huffman wrote:
>>>>>
>>>>>> Hi,
>>>>>> I'm an Arch Linux user and picked up an update a few days ago that=
>>>>> broke
>>>>>
>>>>>> dialyzer. I bisected the last few days of updates and then narrowe=
d
>>>>> the
>>>>>
>>>>>> problem to zlib 1.2.10, which was released January 2nd. 1.2.11 was=
>>>>>> released on the 15th as an emergency bug fix and does not fix the
>>>>>> problem. Reverting my system back to 1.2.8 (the previous version
>>>>>> packaged for Arch) did resolve the issue.
>>>>>> It seems doubtful this is an Erlang problem, but I doubt I'm going=
to
>>>>>> write a test program to demonstrate the problem to them. I though=
t I
>>>>>> should at least report the issue in case others encounter it.
>>>>>> To reproduce, one would need only install zlib 1.2.10 and then run=
:
>>>>>> dialyzer --verbose --build_plt --apps erts --output_plt test.plt
>>>>>> Output would be along the lines of:
>>>>>> dialyzer: Could not get abstract code for file:
>>>>>> /usr/lib/erlang/lib/erts-8.2/ebin/erlang.beam (please recompile it=
>>>>> with
>>>>>
>>>>>> +debug_info)
>>>>>> There are also errors when simply trying to do success typing anal=
ysis
>>>>>> *using* any pre-existing PLT file, along lines of "this isn't a PL=
T
>>>>>> file". The errors are not dependent upon the version of Erlang
>>>>> installed
>>>>>
>>>>>> - at least anything I tried that was released on Arch in the 19.x
>>>>> branch
>>>>>
>>>>>> will reproduce the problem.
>>>>>> Anyway, I hope this report helps someone and I would be curious if=
>>>>>> anyone else reproduces it, or especially if they fail to reproduce=
it.
>>>>>
>>>>>
>>>>> Earlier today (yesterday?), there was the following question on the=
>>>>>
>>>>> erlang-questions mailing list:
>>>>>
>>>>>
>>>>>
>>>>> http://erlang.org/pipermail/erlang-questions/2017-January/0
>>>>> 91434.html
>>>>>
>>>>>
>>>>>
>>>>> I am willing to bet that problem with binary_to_term is also caused=
by
>>>>>
>>>>> zlib troubles.
>>>>>
>>>>>
>>>>>
>>>>> Perhaps Michel (cc:) can inform us about his zlib version.
>>>>>
>>>>>
>>>>>
>>>>> Kostis
>>>>>
>>>>>
>>>
>>> --
>>> Michel Almada de Castro Boaventura
>>> Analista de Sistemas
>>> Laborat=F3rio de Software Livre - LSL
>>>
>>
>>
>> --
>> Michel Almada de Castro Boaventura
>> Analista de Sistemas
>> Laborat=F3rio de Software Livre - LSL
>>
>
>
> _______________________________________________
> erlang-bugs mailing list
> [email protected]
> http://erlang.org/mailman/listinfo/erlang-bugs
--------------DB52A237CC90BD817F0E4DA8
Content-Type: text/html; charset="windows-1252"
Content-Transfer-Encoding: quoted-printable
<html>
<head>
<meta content=3D"text/html; charset=3Dwindows-1252"
http-equiv=3D"Content-Type">
</head>
<body text=3D"#000000" bgcolor=3D"#FFFFFF">
This is indeed a problem in Erlang VM code (shallow copy of inflate
state)<br>
that has existed since R16B03, but not caused actual problem until
zlib v1.2.9.<br>
<br>
Fix coming up. Here is a preliminary patch for the impatient.<br>
<br>
diff --git a/erts/emulator/beam/external.c
b/erts/emulator/beam/external.c<br>
index beed847..1c4fff5 100644<br>
--- a/erts/emulator/beam/external.c<br>
+++ b/erts/emulator/beam/external.c<br>
@@ -1431,6 +1431,10 @@ static B2TContext*
b2t_export_context(Process* p, B2TContext* src)<br>
=A0=A0=A0=A0 if (ctx->state >=3D B2TDecode && ctx->u=
=2Edc.next
=3D=3D &src->u.dc.res) {<br>
=A0=A0=A0=A0=A0=A0=A0=A0 ctx->u.dc.next =3D &ctx->u.dc.res;=
<br>
=A0=A0=A0=A0 }<br>
+=A0=A0=A0 else if (ctx->state =3D=3D B2TUncompressChunk) {<br>
+=A0=A0=A0=A0=A0=A0=A0 int cres =3D inflateCopy(&ctx->u.uc.str=
eam,
&src->u.uc.stream);<br>
+=A0=A0=A0=A0=A0=A0=A0 ASSERT(cres =3D=3D Z_OK); (void)cres;<br>
+=A0=A0=A0 }<br>
=A0=A0=A0=A0 hp =3D HAlloc(p, PROC_BIN_SIZE); <br>
=A0=A0=A0=A0 ctx->trap_bin =3D erts_mk_magic_binary_term(&hp,
&MSO(p), context_b);<br>
=A0=A0=A0=A0 return ctx;<br>
<br>
<br>
/Sverker, Erlang/OTP<br>
<br>
<br>
<div class=3D"moz-cite-prefix">On 01/20/2017 02:49 AM, Jeremy Huffman=
wrote:<br>
</div>
<blockquote
cite=3D"mid:CAH37v0=3D=3D2nquw9J9y+htmckHCfjpnxXuG+HF0mZuzeU4Q2kGVQ@mail.=
gmail.com"
type=3D"cite">
<pre wrap=3D"">I opened a Github issue with zlib. <a class=3D"moz-t=
xt-link-freetext" href=3D"https://github.com/madler/zlib/issues/206">http=
s://github.com/madler/zlib/issues/206</a>.
Mark Adler (zlib maintainer's) response:
"Isolating it to that commit points to a problem in the application code,=
where it must be inadvertently stomping on the deflate state, e.g. with a=
n
out-of-bounds write into memory, or perhaps that the code is trying to us=
e
the deflate state after it has been closed. The only change that commit
made was to check the integrity of the deflate structure more thoroughly =
on
each call of a deflate* function."
On Thu, Jan 19, 2017 at 2:11 PM, Michel Boaventura <
<a class=3D"moz-txt-link-abbreviated" href=3D"mailto:michel.boaventura@gm=
ail.com">[email protected]</a>> wrote:
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">Hi,
I've done the bisect and find the culprit: <a class=3D"moz-txt-link-freet=
ext" href=3D"https://github.com/">https://github.com/</a>
madler/zlib/commit/b516b4bdd7c0c9f0858adfebf732089014f7b282. Before this
commit term_to_binary works and stop doing so afterwards. I will have a
look at the changes and see if I can figure out what happened.
Cheers,
On 19 January 2017 at 16:15, Michel Boaventura <
<a class=3D"moz-txt-link-abbreviated" href=3D"mailto:michel.boaventura@gm=
ail.com">[email protected]</a>> wrote:
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">Hi all,
I'm indeed using zlib 1.2.11 on my gentoo. I can't downgrade it, since
all the other versions were removed from portage.
I will clone zlib repo and see if I can bisect the problem.
Thanks!
On 19 January 2017 at 15:45, Jeremy Huffman <a class=3D"moz-txt-link-rfc2=
396E" href=3D"mailto:[email protected]"><[email protected]=
om></a>
wrote:
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">Yes it's exactly the same error message from d=
ialyzer. And the fact that
he's getting it on Gentoo which builds from source suggests that it is no=
t
simply a matter of recompiling the dependency chain, which was a suggesti=
on
in the Arch board. There was another app in Arch that also had a problem
pinned on zlib 1.2.11.
On Thu, Jan 19, 2017 at 11:33 AM Kostis Sagonas <a class=3D"moz-txt-link-=
rfc2396E" href=3D"mailto:[email protected]"><[email protected]></a>=
wrote:
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">On 01/19/2017 03:42 AM, Jeremy Huffman wrote=
:
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">Hi,
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">I'm an Arch Linux user and picked up an up=
date a few days ago that
</pre>
</blockquote>
<pre wrap=3D"">broke
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">dialyzer. I bisected the last few days of =
updates and then narrowed
</pre>
</blockquote>
<pre wrap=3D"">the
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">problem to zlib 1.2.10, which was released=
January 2nd. 1.2.11 was
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">released on the 15th as an emergency bug f=
ix and does not fix the
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">problem. Reverting my system back to 1.2.8=
(the previous version
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">packaged for Arch) did resolve the issue.
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">It seems doubtful this is an Erlang proble=
m, but I doubt I'm going to
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">write a test program to demonstrate the pr=
oblem to them. I thought I
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">should at least report the issue in case o=
thers encounter it.
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">To reproduce, one would need only install =
zlib 1.2.10 and then run:
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">dialyzer --verbose --build_plt --apps erts=
--output_plt test.plt
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">Output would be along the lines of:
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">dialyzer: Could not get abstract code for =
file:
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">/usr/lib/erlang/lib/erts-8.2/ebin/erlang.b=
eam (please recompile it
</pre>
</blockquote>
<pre wrap=3D"">with
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">+debug_info)
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">There are also errors when simply trying t=
o do success typing analysis
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">*using* any pre-existing PLT file, along l=
ines of "this isn't a PLT
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">file". The errors are not dependent upon t=
he version of Erlang
</pre>
</blockquote>
<pre wrap=3D"">installed
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">- at least anything I tried that was relea=
sed on Arch in the 19.x
</pre>
</blockquote>
<pre wrap=3D"">branch
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">will reproduce the problem.
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">Anyway, I hope this report helps someone a=
nd I would be curious if
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<blockquote type=3D"cite">
<pre wrap=3D"">anyone else reproduces it, or especially i=
f they fail to reproduce it.
</pre>
</blockquote>
<pre wrap=3D"">
Earlier today (yesterday?), there was the following question on the
erlang-questions mailing list:
<a class=3D"moz-txt-link-freetext" href=3D"http://erlang.org/pipermail=
/erlang-questions/2017-January/0">http://erlang.org/pipermail/erlang-ques=
tions/2017-January/0</a>
91434.html
I am willing to bet that problem with binary_to_term is also caused by
zlib troubles.
Perhaps Michel (cc:) can inform us about his zlib version.
Kostis
</pre>
</blockquote>
</blockquote>
<pre wrap=3D"">
--
Michel Almada de Castro Boaventura
Analista de Sistemas
Laborat=F3rio de Software Livre - LSL
</pre>
</blockquote>
<pre wrap=3D"">
--
Michel Almada de Castro Boaventura
Analista de Sistemas
Laborat=F3rio de Software Livre - LSL
</pre>
</blockquote>
<pre wrap=3D"">
</pre>
<br>
<fieldset class=3D"mimeAttachmentHeader"></fieldset>
<br>
<pre wrap=3D"">_______________________________________________
erlang-bugs mailing list
<a class=3D"moz-txt-link-abbreviated" href=3D"mailto:[email protected]=
rg">[email protected]</a>
<a class=3D"moz-txt-link-freetext" href=3D"http://erlang.org/mailman/list=
info/erlang-bugs">http://erlang.org/mailman/listinfo/erlang-bugs</a>
</pre>
</blockquote>
<br>
</body>
</html>
--------------DB52A237CC90BD817F0E4DA8--
--===============0543584806586181690==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
erlang-bugs mailing list
[email protected]
http://erlang.org/mailman/listinfo/erlang-bugs
--===============0543584806586181690==--