Re: Value and Principles of Unit Testing.

--- John Carter <john.carter@...> wrote:
> ... that makes me all the more determine to get it Right!

There is no "right." We strive to do a good job. But there is always room for improvement.

You try your best ideas with the team. And then you, as a team, improve them.

> The large body of embedded C software they are working on,
> has slowly being growing unit test coverage and has now
> reached around 16% by SLOC.
> ...
> I been cringing as I wade through the list of "test smells",
> I recognize them all.

First:  We live, we learn.

No matter how well you have done, tomorrow you will know more and be able to do more. You will look back and see areas where you can improve.

Recognize the accomplishments you have achieved. For example, most projects have no tests. You're all the way up to 16%. You have started, and that's the hardest part.


[I cut a lot of really good stuff from the message here. It's great stuff, so I don't have much to say about it.]


> The larger part of production code under test, the weaker your test
> coverage is and the more fragile your test is.
> Minimize Test Overlap, Verify One Condition Per Test.

But remember: If the parts don't work together properly, the system as a whole fails. There is value to testing large integrated chunks, too.

> There should only one reason for a test to fail (Defect
> Localization) and only one test to fail due to a defect
> (Effective, Targeted Testing).

> In future we should aim to be testing, in each test, a
> single, specific aspect of the Code Under Test. Ideally,
> if that single aspect is broken, only one test should
> fail, and that should be the only reason that that test
> could fail.

Good ideas. Try not to stress about them too much.


> Keep Test Logic Out of Production Code.

But be open to making reasonable changes to the design of the production code to make it testable.

> These "test the framework tests" need to be actively discarded.

> If you mistrust the framework, add tests to the unit tests for the
> framework.

But it is reasonable to test that you are using the functionality of the framework correctly to achieve the desired result.

> "const" is a keyword that always make me relax and feel
> less stressed. "const" is a remarkable powerful statement.
> Use it where ever possible.

["Const correctness" is a religion. And a virus. If/when you adopt it, it will spread throughout your code base. This can be a very good thing. Just don't be surprised at its viral nature.]

> I/O functions are hard because...
...
>  To cope with these facts, we need to alter our designs to make them
> testable. (Turns out this is actually A Good Thing!)
...

Yes, testing I/O is hard -- user I/O and interfaces to complex external systems that are non-trivial to control.

Isolate and mock them.


> Conclusions
>    - As our Test Coverage has grown, "smells" and weaknesses
>      in our tests have emerged and need to be addressed.
>    - We need to emphasize Design for Test.
>    - We need to highlight the differences between pure,
>      stateful, service and I/O functions and adjust our test
>      strategies accordingly.
>    - Unit Tests are about Defect Localization, not paranoia.

Looks like a good list. Lots of good ideas.


An agile approach is for you and your team to work together to improve this over time.



------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/testdrivendevelopment/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/testdrivendevelopment/join
    (Yahoo! ID required)

<*> To change settings via email:
    [email protected] 
    [email protected]

<*> To unsubscribe from this group, send an email to:
    [email protected]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/