Re: easier debugging of unit tests

On Wed, 2014-04-02 at 13:10 +0200, Jan Wielemaker wrote:
> On 04/01/2014 08:01 PM, Ross Boylan wrote:
> > A few questions.
> > 1. guitracer. trace. run_tests('disk rules':vl1).
> >
> > takes me into the debugger and I need to step through all the testing
> > machinery, including matching the right test, before I get to my code.
> > Is there a better way?
> 
> I've added a few declarations and did some small changes to
> hide most of the internals now.  That was always intended,
> but got a bit rusty.
Will those just be in 7.x series?  Is there a path somewhere I could use
to modify my local copy?
> 
> > I tried putting stops on various points in
> > matching_test(X, X) :- !.
> > matching_test(Name, Set) :-
> > 	is_list(Set),
> > 	memberchk(Name, Set).
> >
> > and using continue to breakpoint, but this either yields the same number
> > of stops as before or no stops at all.  In particular, putting a stop
> > before the first line doesn't cause a stop.
> 
> Would need real details to comment.
I'm not sure what other details I can provide/you want.
I start the debugging as shown in the top line, hit space a few times
and end up in the matching_test code (would "clauses" be a more prolog-y
term?).  Then I position the cursor at the start of the line, e.g., for
the matching_tests(X, X) condition and and click on the stop icon.  A
top icon then appears to the left of the line in the debugger.  The I
type l or hit the corresponding icon and the test runs to completion and
returns the the non-gui command line.

I have tried positioning the breakpoint before various of the lines.
For some of them in match_test(Name, Set) the breakpoint does cause
execution to stop after I hit l; for others it seems to do nothing.  I
also tried positioning the cursor just before the ! in the
matching_test(X,X) clause.  That also had no effect.  Then I tried
positioning the cursor immediately after the !, hit the stop button and
got 
ERROR: '$break_at'/3: No permission to set break `break(528820,4)'
ERROR: '$break_at'/3: No permission to set break `break(528820,4)'
ERROR: '$break_at'/3: No permission to set break `break(528820,4)'
ERROR: '$break_at'/3: No permission to set break `break(528820,4)'
ERROR: '$break_at'/3: No permission to set break `break(528820,4)'
ERROR: '$break_at'/3: No permission to set break `break(528820,4)'
ERROR: '$break_at'/3: No permission to set break `break(528820,4)'
ERROR: '$break_at'/3: No permission to set break `break(528820,4)'
ERROR: '$break_at'/3: No permission to set break `break(528820,4)'
ERROR: '$break_at'/3: No permission to set break `break(528820,4)'
ERROR: '$break_at'/3: No permission to set break `break(528820,4)'
Foreign predicate system:$free_variable_set/3 did not clear exception: error(permission_error(set,break,break(528820,4)),context(system: $break_at/3,_G2724))

|  
in my non-GUI terminal.
> 
> > 2. Is there a way to proceed to the next line on the same level?  In
> > other words, given I just evaluated the first line of
> > pred1(A),
> > pred2(B, A),
> > pred3(C).
> > can I have the system evaluate pred2 without the debugger stepping into
> > it?
> 
> Use 's' (skip).  There is also an icon for it.
For some reason I thought that was skipping the evaluation entirely, but
I just tried it and it doesn't.  Thanks.
> 
> > Two work-arounds are to step into pred2 and then use finish, or put a
> > stop before pred3 and continue to breakpoint.  I'm hoping there's a
> > better way.
> >
> > 3. Sometimes, particularly after I execute make, the tests seem to lose
> > the ability to see the definitions they need (both within the debugger
> > and doing run_tests without the debugger).  I think the problem is that
> > the facts it needs are inside a namespace that the test loses access to
> > (even though the test is in the same namespace, I think).  Is there a
> > solution?
> >
> > The tests are in rulesTest.plt.  That program starts
> > :- begin_tests('disk rules').
> > :- ['rules.pl'].
> >
> > Then it has the facts to which I lose access, and then the unit tests.
> 
> Tests where surely not meant to be defined by consulting a file
> between begin/end tests.  Why do you try that? 
The tests themselves are not in rules.pl; that has the logic I am trying
to test.

rulesTest.plt looks like
:- begin_tests('disk rules').
:- ['rules.pl'].

<a bunch of facts/statements>
<actual tests like 
test(bl63, [set(r(D, DW) = [r(encrypt(lvlvold_mynewvg), F1/1/21)])]) :-
	   block(lvlvold_mynewvg, F1/1/21, D, DW).
>
:- end_tests('disk rules').

It is the material in <a bunch of facts> that seems to become
inaccessible sometimes after make.

What other route do you suggest I take?  Both the rules and the tests
are large enough that I'd rather not have them in one file.

>  If you really want,
> :- include(rules). might work, but probably only in recent versions.
Meaning 7.x?
> 
> As is, make/0 will reload rules.pl if it was modified, but unaware of
> the fact that it appears in a testing environment.
What happens then?  Do I end up with 2 separate sets of definitions of
the rules in different scopes?

I guess if the test (e.g., bl63 above) ends up calling the globally
scoped block/4, and this globally scoped block/4 can not see the facts
declared inside the test scope that might explain what I'm seeing.

I'm unclear about exactly what is going on with the scoping inside the
tests.  Sometimes, such as when I invoke the test, I need to scope by
'disk rules'.  But some output shows 'plunit_disk rules', e.g.,
% Breakpoint 1 in 6-th clause of 'plunit_disk rules':block/4 at rules.pl:677

I have tried placing breakpoints like
guitracer. debug('disk rules':block/4). trace. run_tests('disk rules':bl11).
but it seems ineffective (also tried with no scope and 'plunit_disk rules').

Ross
> 
> 	Cheers --- Jan
> 
> >
> > Prolog 6.4.1.
> >
> > Thanks.
> > Ross Boylan
> >
> >
> >
> >
> > _______________________________________________
> > SWI-Prolog mailing list
> > [email protected]
> > https://lists.iai.uni-bonn.de/mailman/listinfo.cgi/swi-prolog
> >
>