Re: ASDF test-op

Todd Sabin wrote:
> Robert Goldman <[email protected]> writes:
>> Todd Sabin wrote:
>>> Hi,
>>>
>>> Doesn't ASDF already write all of its output to a particular stream,
>>> namely *verbose-out*?  (Provided you've bound it to something, of
>>> course) Couldn't you just document that test-op writers should write
>>> to that stream?  Or maybe that doesn't work because, by default, it's
>>> bound to nil, so you define another dynamic variable *test-output*, or
>>> something, and document that?
>>>
>>> I guess I don't really understand the problem you're solving, and I'm
>>> wondering why all the CLOS machinery is necessary...
>>>
>> What we're trying to avoid is having the test outputs be mixed in with
>> the normal ASDF outputs.
>>
>> Here's a concrete bad (by "bad" I mean "undesirable," not "buggy")
>> behavior I have seen:
>>
>> I have a test-op for a large and complex system.  This test op loads
>> multiple ASDF components, each of which gets compiled.
>>
>> The problem is that instead of getting a nice printed output from my
>> testing, I get a stew of compiler messages and test outputs all mooshed
>> together.  This makes it hard for me to see if the tests went ok.
>>
>> So I'd like to send the test outputs off somewhere else, so I don't have
>> to skip over all the compiler messages.
> 
> Why doesn't something like
> 
> (defvar *test-op-output* *standard-output*
>   "The stream to which test-op operations should write their output")
> 
> suffice?  You would, of course, bind or assign *test-op-output* to
> something of your liking.

Well, for one thing, because I don't want every unit test  library in
the world to have to know that it should be consulting this global
variable.  The various unit test libraries should be able to do their
own thing in their own way.  We (the ASDF community) should just be
making it possible for the test library to hook in.

If you start to think about how to write :around methods for various
bits of ASDF in order to get the information about *test-op-output* into
the unit test libraries, I think you'll see that there's a fair amount
of hair involved.

Further, it involves forcing people to write

(let ((asdf:*test-op-output* my-stream))
  (asdf:oos 'asdf:test-op my-library))

and I'd prefer

(asdf:oos 'asdf:test-op my-library :output-stream <foobar>)

[As an aside, I'm inclined to think we should really have a separate
:test-output-stream, or some way to make it possible for the coder to
know that not ALL of the output should go to this stream.]
> 
> Defining the generic functions, etc., that were proposed seems to
> raise a lot more questions, to me.  E.g., ASDF already has
> *verbose-out*, though on a closer look, it doesn't seem to be intended
> for the user to manipulate directly.  Instead, there's the :verbose
> argument to OPERATE.  Should *verbose-out* go away, and everywhere
> that ASDF currently writes to it be changed to write to
> (output-stream-for-operation-component op c) ?  And if you add
> :output-stream as an argument to OPERATE, shouldn't :verbose go away?

I don't think so (although this is a different issue) --- :verbose is a
question of how much output and *verbose-out* is the destination for the
output, or at least it should be.
> 
> It also seems like slots on the operation and initargs will make
> things not work the way you want.  If you have in your system
> definition
> 
>    :in-order-to ( ;;; etc 
>                  (test-op (compile-op my-system)
>                           (load-op my-system)) ;; or whatever the syntax is
> 
> when ASDF creates the child operations to compile and load the system,
> won't those operations will get the same :output-stream initarg, and
> then you still get the compiler output in with your test output?

I agree that this should not be an :output-stream -- it should really be
a test-output stream, see above.

but if you look into the guts of ASDF, it seems that some thought was
put to how to make the initargs be inherited.


Best,
r



-- 
Robert P. Goldman
Senior Scientist
Smart Information Flow Technologies (d/b/a SIFT, LLC)

211 N. First St., Suite 300
Minneapolis, MN 55401

Voice:	(612) 384-3454
Email:    [email protected]

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/