Re: Bugs reported because of failing tests more likely to be reopened?

--047d7bb04f681896d004ebc37a90
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Yeah, I was reading reopened TEST bug reports and it seems that, indeed,
randomness and differences in platform and operation system are common
reasons to reopen a TEST bug.

I'm still not sure how to interpret the result, though. Surely, the fact
that the bug was discovered because of a failing test does not *cause* the
reopening afterwards. But why would TEST bugs be more likely to have these
problems with platform/OS than regular bugs? I guess I'll try to analyse
other hypotheses about bug reopening.

Thanks again for your feedback, you're the best!

[]s
Rodrigo

On Fri, Nov 22, 2013 at 5:55 AM, Ji=C5=99=C3=AD Kovalsk=C3=BD <jiri.kovalsk=
[email protected]>wrote:

> +1
>
> Best regards,
> -Jirka
>
>
> On 22.11.2013 09:44, Jiri Skrivanek wrote:
>
>  One possible reason can be that responsible developers fix failing test
>> on their primary platform/JDK and when you run test on multiple
>> operating systems and JDKs it fails again.
>>
>> Jiri
>>
>> On 22.11.2013 0:54, Rodrigo Rocha Gomes e Souza wrote:
>>
>>> Hello again,
>>>
>>> I was doing some statistical analyses with NetBeans Platform's bug
>>> reports and concluded that...
>>>
>>> ... bugs related to failing, unfinishing or errors in the test (i.e.,
>>> bugs with keyword TEST) are 2.5x more likely to be reopened after being
>>> fixed.
>>>
>>> Do you have any theories that explain this result?
>>>
>>> The result was surprising for me, because I thought it would be the
>>> other way around: bugs reported because of a failing test would be
>>> *less* likely to be reopened because they should be easier to reproduce=
.
>>> After all, in this case the developer himself can verify if the fix was
>>> appropriate just by running the failing test. What do you think?
>>>
>>> []s
>>> Rodrigo
>>> Federal University of Bahia
>>>
>>

--047d7bb04f681896d004ebc37a90
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Yeah, I was reading reopened TEST bug reports and it seems=
 that, indeed, randomness and differences in platform and operation system =
are common reasons to reopen a TEST bug.<br><div class=3D"gmail_extra"><br>=
</div>


<div class=3D"gmail_extra" style>I&#39;m still not sure how to interpret th=
e result, though. Surely, the fact that the bug was discovered because of a=
 failing test does not *cause* the reopening afterwards. But why would TEST=
 bugs be more likely to have these problems with platform/OS than regular b=
ugs? I guess I&#39;ll try to analyse other hypotheses about bug reopening.<=
/div>

<div class=3D"gmail_extra" style><br></div><div class=3D"gmail_extra" style=
>Thanks again for your feedback, you&#39;re the best!</div><div class=3D"gm=
ail_extra" style><br></div><div class=3D"gmail_extra" style>[]s</div><div c=
lass=3D"gmail_extra" style>

Rodrigo</div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On F=
ri, Nov 22, 2013 at 5:55 AM, Ji=C5=99=C3=AD Kovalsk=C3=BD <span dir=3D"ltr"=
>&lt;<a href=3D"mailto:[email protected]" target=3D"_blank">jiri.kov=
[email protected]</a>&gt;</span> wrote:<br>



<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">+1<br>
<br>
Best regards,<br>
-Jirka<div><div><br>
<br>
On 22.11.2013 09:44, Jiri Skrivanek wrote:<br>
<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
One possible reason can be that responsible developers fix failing test<br>
on their primary platform/JDK and when you run test on multiple<br>
operating systems and JDKs it fails again.<br>
<br>
Jiri<br>
<br>
On 22.11.2013 0:54, Rodrigo Rocha Gomes e Souza wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
Hello again,<br>
<br>
I was doing some statistical analyses with NetBeans Platform&#39;s bug<br>
reports and concluded that...<br>
<br>
... bugs related to failing, unfinishing or errors in the test (i.e.,<br>
bugs with keyword TEST) are 2.5x more likely to be reopened after being<br>
fixed.<br>
<br>
Do you have any theories that explain this result?<br>
<br>
The result was surprising for me, because I thought it would be the<br>
other way around: bugs reported because of a failing test would be<br>
*less* likely to be reopened because they should be easier to reproduce.<br=
>
After all, in this case the developer himself can verify if the fix was<br>
appropriate just by running the failing test. What do you think?<br>
<br>
[]s<br>
Rodrigo<br>
Federal University of Bahia<br>
</blockquote></blockquote>
</div></div></blockquote></div><br></div></div>

--047d7bb04f681896d004ebc37a90--