test/subtest flow control with exceptions

--oyUTqETQ0mS9luUI
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Often, I have a test like this:

  subtest "do things with an api" =3D> sub {
    my $result =3D $api_client->do_first_thing;

    is(
      $result->documents->first->title,
      "The Best Thing",
    );

    ...
  };

Sometimes, the result comes back with zero documents.  ->first throws an
exception and then my whole test program comes crashing down and it's
miserable.  For a while, I've been meaning to make it possible for some
exceptions to be recognized by my test programs as instructions to emit a
failure, stop this subtest, and move on.

It's important to note that I'm talking, in the code above, about an except=
ion
that would be thrown by ($result->documents) when ->first is called on it.
This kind of flow control eliminates needing to write:

  my $result =3D $api_client->do_first_thing;
  my $docs   =3D $result->documents;
  fail("no docs"), return unless $docs->has_entries;
  my $first  =3D $docs->first;
  fail("no title"), return unless $first->has_title;

  is(...);

In my case, I'm working with an API client that's specifically designed to =
be
used for testing, so this kind of loose coupling between thrown exceptions =
and
the test code is a good fit.  Not every exception should be caught this way.
Truly unexpected ones should still die.

So, I've written something to do this, and I'm sharing it here before going
further with it.  In my code, if an exception is meant to be caught and use=
d as
a local abort instruction, it has a method called as_test_abort_events.  Th=
is
method returns a reference to an array of Test2 event descriptions.  For
example, maybe:

  sub as_test_abort_events {
    return [
      [ Ok   =3D> (pass =3D> 0, name =3D> "no documents, but ->first called=
") ],
      [ Diag =3D> (message =3D> "collection state: ....") ],
    ];
  }

This method is easy to add to any exception you want, and you don't need to
change your exception class hierarchy in any way.

Next, you need something to run the tests and look for these exceptions bei=
ng
thrown.  I have written a library for this, called Test::Abortable.

  https://github.com/rjbs/Test-Abortable

Test::Abortable provides two subroutines: subtest and testeval.  subtest ac=
ts
just like Test::More's subtest, but catches abort exceptions, emits their
events, and returns normally.  (I've also updated my library Test::Routine,=
 in
a branch, to behave this way, as I use it in place of subtest for many thin=
gs.)

testeval acts like eval, but only catches abort exceptions.

  ok(1);
  testeval {
    ok(2);
    this_throws_an_abort;
    ok(3);
  };
  ok(4);

This ends up emitting ok 1, ok 2, whatever the abort wants, and ok 4.  test=
eval
returns the return value of the code block if it succeeds.  If it fails due=
 to
abort, it returns false, emits the abort events, and puts the abort in $@. =
 If
it fails because of any other exception, the exception is re-thrown.

Let me know if you have any thoughts before I begin using this in anger. :-)

--=20
rjbs

--oyUTqETQ0mS9luUI
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJYPjDbAAoJEOYby6cMccU5LGsIALZG5liA9lMeWeJTBnFTAaR3
lwZ1dgioIZTNBwQBFBqoAa4ORIrsInOpfaYT0c4nsCekGWgDILzH4m8zAOgnfrTj
PiwRfa2kk4l+K+niSLqBnoaGDOwOpwpw7J4x2nvBkSO5YygIsDRVQI9Ug6OO4aZX
/R2GLj0+6DlMiKakfRjLp4XnH2FO5RxU16i5fFyIYt5u2BTvr082dPuixJ0aOV0A
a5P+KisRl8zYPgkvugusT/Iw/KNXNbiWomcG8OuRSepFLWqKAC9IIAexNT61ZykC
OZ60EDbpV/S3h0ul1+rIfgMeGjoASviEgecWlawe6VWCok293BBK+m6yRmZMwlg=
=1PPG
-----END PGP SIGNATURE-----

--oyUTqETQ0mS9luUI--