Re: CPANifying our test framework - or parts of it

--001a113d6dda41d670053c1ddd78
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

You might find Test::Deep right up your alley!

On Fri, Sep 9, 2016 at 6:34 PM, Sam Kington <[email protected]> wrote:

> Hi,
>
> At $WORK we have an extensive test suite that we=E2=80=99ve built up over=
 the
> years, with loads of convenience methods for calling e.g. Dancer endpoint=
s
> and checking returned data structures against what we expect. One of our
> dev team recently left for pastures new, and would like to carry on using
> these tools. How should I best extract this functionality into a proper
> CPAN distribution (ideally using Test2)?
>
> At its most elaborate, our current test code lets you say e.g.
>
> # Assume $self->state('location_id') has been set previously
> $self->test(
>     title =3D> 'Create a properly megalithic structure',
>     call  =3D> [
>         POST =3D> '/location/:location_id/build',
>         {
>             author     =3D> 'Wally Wallington',
>             structures =3D> [
>                 {
>                     type     =3D> 'dolmen',
>                     material =3D> 'concrete',
>                 }
>             ]
>         }
>     ],
>     expect =3D> {
>         http_code =3D> HTTP_CREATED,
>         sql_count =3D> 12,
>         data      =3D> {
>             location_id =3D> $self->state('location_id'),
>             build_id    =3D> qr{^ (?<build_id> BUILD \d+ ) $}x,
>             built       =3D> $self->true,
>             author     =3D> $self->meh,    # It's OK if they call him Idi=
ot.
>             structures =3D> [
>                 {
>                     _hashref_contains =3D> {
>                         structure_id =3D> qr{^ (?<structure_id> STRUCT \d=
+ )
> $}x,
>                         type         =3D> 'dolmen',
>                         material     =3D> 'concrete',
>
>                         # There's probably stuff about where the dolmen
>                         # was erected but we ignore that for the purpose
>                         # of this test.
>                     }
>                 }
>             ]
>         }
>     }
> );
>
> # $self->state('build_id') got set by the named capture in the regex abov=
e.
> $self->test(
>     title =3D> 'Turn it to gold because lol',
>     call  =3D> [
>         PATCH =3D> '/location/:location_id/build/:build_id/structure/:
> structure_id',
>         { material =3D> 'gold' }
>     ],
>     expect =3D> {
>         # The same as saying data =3D> { _hashref_contains =3D> { materia=
l =3D>
> 'gold' } }
>         data_contains =3D> {
>             material =3D> 'gold'
>         }
>     }
> );
>
> And if anything doesn=E2=80=99t quite match, it tries to show you what di=
d and
> didn=E2=80=99t match.
>
> The stuff in call gets passed to e.g. Dancer::Test::dancer_response or
> treated as a method call, depending on the invocation. As we share a comm=
on
> process with the code being called and tested, we can also monitor the
> database calls being made via DBIx::Class, and there=E2=80=99s some fiend=
ish code
> for tracking, diagnosing and correcting test errors regarding the number =
of
> distinct database queries being made.
>
> These are all nice-to-haves rather than something that=E2=80=99s inherent=
 to the
> design of this testing code. What=E2=80=99s really useful, and what my ex=
-colleague
> is hankering after, is a couple of things:
>
> (A) To say, flexibly, =E2=80=9CI want a data structure that looks like th=
is=E2=80=9D, and
> to be able to say (1) this field must look exactly like this, (2) this
> field should look like this, and remember it, (3) this field should exist=
,
> but I don=E2=80=99t care what its value is, and (4) there might be additi=
onal
> fields but I don=E2=80=99t care about them for the purpose of this test.
>
> (B) To maintain a collection of placeholders that can be updated by tests
> and used by subsequent tests.
>
> I haven=E2=80=99t looked recently at the Test:: infrastructure, so for al=
l I know
> we may have replicated functionality that we=E2=80=99d missed a few years=
 ago when
> we started coding.
>
> But if this functionality is new and useful, how should I implement it?
>
> Sam
> --
> Website: http://www.illuminated.co.uk/
>
>

--001a113d6dda41d670053c1ddd78
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_default" style=3D"font-size:large">You=
 might find Test::Deep right up your alley!<br></div></div><div class=3D"gm=
ail_extra"><br><div class=3D"gmail_quote">On Fri, Sep 9, 2016 at 6:34 PM, S=
am Kington <span dir=3D"ltr">&lt;<a href=3D"mailto:[email protected]" t=
arget=3D"_blank">[email protected]</a>&gt;</span> wrote:<br><blockquote=
 class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc soli=
d;padding-left:1ex">Hi,<br>
<br>
At $WORK we have an extensive test suite that we=E2=80=99ve built up over t=
he years, with loads of convenience methods for calling e.g. Dancer endpoin=
ts and checking returned data structures against what we expect. One of our=
 dev team recently left for pastures new, and would like to carry on using =
these tools. How should I best extract this functionality into a proper CPA=
N distribution (ideally using Test2)?<br>
<br>
At its most elaborate, our current test code lets you say e.g.<br>
<br>
# Assume $self-&gt;state(&#39;location_id&#39;) has been set previously<br>
$self-&gt;test(<br>
=C2=A0 =C2=A0 title =3D&gt; &#39;Create a properly megalithic structure&#39=
;,<br>
=C2=A0 =C2=A0 call=C2=A0 =3D&gt; [<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 POST =3D&gt; &#39;/location/:location_id/build&=
#39;<wbr>,<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 {<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 author=C2=A0 =C2=A0 =C2=A0=3D&gt;=
 &#39;Wally Wallington&#39;,<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 structures =3D&gt; [<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 {<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 type=
=C2=A0 =C2=A0 =C2=A0=3D&gt; &#39;dolmen&#39;,<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 mater=
ial =3D&gt; &#39;concrete&#39;,<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 }<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ]<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 }<br>
=C2=A0 =C2=A0 ],<br>
=C2=A0 =C2=A0 expect =3D&gt; {<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 http_code =3D&gt; HTTP_CREATED,<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 sql_count =3D&gt; 12,<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 data=C2=A0 =C2=A0 =C2=A0 =3D&gt; {<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 location_id =3D&gt; $self-&gt;sta=
te(&#39;location_id&#39;),<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 build_id=C2=A0 =C2=A0 =3D&gt; qr{=
^ (?&lt;build_id&gt; BUILD \d+ ) $}x,<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 built=C2=A0 =C2=A0 =C2=A0 =C2=A0=
=3D&gt; $self-&gt;true,<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 author=C2=A0 =C2=A0 =C2=A0=3D&gt;=
 $self-&gt;meh,=C2=A0 =C2=A0 # It&#39;s OK if they call him Idiot.<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 structures =3D&gt; [<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 {<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 _hash=
ref_contains =3D&gt; {<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 structure_id =3D&gt; qr{^ (?&lt;structure_id&gt; STRUCT \d+ ) $}=
x,<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 type=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=3D&gt; &#39;dolmen&#39;,<=
br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 material=C2=A0 =C2=A0 =C2=A0=3D&gt; &#39;concrete&#39;,<br>
<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 # There&#39;s probably stuff about where the dolmen<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 # was erected but we ignore that for the purpose<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 # of this test.<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 }<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 }<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 ]<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 }<br>
=C2=A0 =C2=A0 }<br>
);<br>
<br>
# $self-&gt;state(&#39;build_id&#39;) got set by the named capture in the r=
egex above.<br>
$self-&gt;test(<br>
=C2=A0 =C2=A0 title =3D&gt; &#39;Turn it to gold because lol&#39;,<br>
=C2=A0 =C2=A0 call=C2=A0 =3D&gt; [<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 PATCH =3D&gt; &#39;/location/:location_id/build=
/<wbr>:build_id/structure/:<wbr>structure_id&#39;,<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 { material =3D&gt; &#39;gold&#39; }<br>
=C2=A0 =C2=A0 ],<br>
=C2=A0 =C2=A0 expect =3D&gt; {<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 # The same as saying data =3D&gt; { _hashref_co=
ntains =3D&gt; { material =3D&gt; &#39;gold&#39; } }<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 data_contains =3D&gt; {<br>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 material =3D&gt; &#39;gold&#39;<b=
r>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 }<br>
=C2=A0 =C2=A0 }<br>
);<br>
<br>
And if anything doesn=E2=80=99t quite match, it tries to show you what did =
and didn=E2=80=99t match.<br>
<br>
The stuff in call gets passed to e.g. Dancer::Test::dancer_response or trea=
ted as a method call, depending on the invocation. As we share a common pro=
cess with the code being called and tested, we can also monitor the databas=
e calls being made via DBIx::Class, and there=E2=80=99s some fiendish code =
for tracking, diagnosing and correcting test errors regarding the number of=
 distinct database queries being made.<br>
<br>
These are all nice-to-haves rather than something that=E2=80=99s inherent t=
o the design of this testing code. What=E2=80=99s really useful, and what m=
y ex-colleague is hankering after, is a couple of things:<br>
<br>
(A) To say, flexibly, =E2=80=9CI want a data structure that looks like this=
=E2=80=9D, and to be able to say (1) this field must look exactly like this=
, (2) this field should look like this, and remember it, (3) this field sho=
uld exist, but I don=E2=80=99t care what its value is, and (4) there might =
be additional fields but I don=E2=80=99t care about them for the purpose of=
 this test.<br>
<br>
(B) To maintain a collection of placeholders that can be updated by tests a=
nd used by subsequent tests.<br>
<br>
I haven=E2=80=99t looked recently at the Test:: infrastructure, so for all =
I know we may have replicated functionality that we=E2=80=99d missed a few =
years ago when we started coding.<br>
<br>
But if this functionality is new and useful, how should I implement it?<br>
<span class=3D"HOEnZb"><font color=3D"#888888"><br>
Sam<br>
--<br>
Website: <a href=3D"http://www.illuminated.co.uk/" rel=3D"noreferrer" targe=
t=3D"_blank">http://www.illuminated.co.uk/</a><br>
<br>
</font></span></blockquote></div><br></div>

--001a113d6dda41d670053c1ddd78--