Re: Single assert / Single mock rule in practice

Here is how I define a unit of work (the thing under test) (from book
chapter PDF:
http://www.manning-source.com/books/osherove2/sample_ch01_Osherove2_November11.pdf)

Definition
A unit of work is the sum of actions that take place between the invocation
of a public
method in the system and a single noticeable end result by a test of that
system. A
noticeable end result can be observed without looking at the internal state
of the system and only through its public APIs and behavior. An end result
is any of the following:
■ The invoked public method returns a value (a function that’s not void).
■ There’s a noticeable change to the state or behavior of the system before
and
after invocation that can be determined without interrogating private state.
(Examples: the system can log in a previously nonexistent user, or the
system’s
properties change if the system is a state machine.)
■ There’s a callout to a third-party system over which the test has no
control, and
that third-party system doesn’t return any value, or any return value from
that
system is ignored. (Example: calling a third-party logging system that was
not
written by you and you don’t have the source to.)


*****END SNIP***

As far as I am concerned, the unit of work could span multiple classes or
methods. if you have a dependency that is closely tied to the thing under
test, and that you can run with you test in memory without faking it, I
would try to not fake it. I would only use a mock where I had no other way
but to prove a collaborator had to be called as an end result. For example,
I would use a STUB for a get_something if not faking it meant an
integration test. I would not use a mock for it, since it is giving info
back into the application.
I would use a mock only if there an is a "tell" operation on a collaborator
as an end result.
stubs: queries
mocks: commands

but only when you have no other choice but to fake it. I would rather test
state or a return value and have only real classes. Mocks make tests
complicated.

My view is the opposite of the "mockist" approach wherein you use mocks to
design the interaction protocol between classes via asserts. I find that
approach leads to very hard to maintain tests over time.


On Wed, Jan 15, 2014 at 9:32 PM, Adam Sroka <[email protected]> wrote:

>
>
> P.S. in the first sentence of that second paragraph I should have said you
> stub the method and assert something about the SUT, only. In other words:
>
> For a query:
> Arrange: stub the query and pass the stub to the object under test
> Act: call the method on the object under test that will cause the stubbed
> method to be queried
> Assert: assert something about the object under test that should only be
> true if the stubbed method returns correctly
>
> For a command:
> Arrange: create a mock and pass it to the object under test *
> Act: call the method on the object under test that will cause the method
> on the mock to be called
>  Assert: verify that the mock was called correctly which should only be
> true if the object under test did what we expected
>
> * depending on the framework you may also have to specify how the mock is
> expected to be called before you call it.
>
>
>
> On Wed, Jan 15, 2014 at 3:20 PM, Adam Sroka <[email protected]> wrote:
>
>> Are you familiar with Command Query Separation (CQS)? I've always found
>> that the most coherent and useful way to explain why you would choose a
>> mock or a stub. The most interesting thing about a method is either what it
>> returns or, in the case where you don't care what it returns, whether it
>> was called with the correct semantics, not both.
>>
>> So, if the SUT cares about the value that a method returns then you stub
>> that method, only. If the SUT doesn't care about the value a method returns
>> then you use a mock and verify the call has the expected semantics, only.
>>
>> Does that help?
>>
>>
>>
>> On Tue, Jan 14, 2014 at 6:37 AM, Mateusz Łoskot <[email protected]>wrote:
>>
>>>
>>>
>>> Hi,
>>>
>>> I'd like to ask for ideas of best practices in case of developing and
>>> testing
>>> a wrapper for a Python legacy module.
>>> Especially, about applying the single assert / single mock rule
>>> which I have been learning about from numerous books/videos/blogs.
>>>
>>> I have Python xdep module which exposes API in form of bunch of free
>>> functions.
>>> I'm creating new Python module foo which uses xdep (the idea is to
>>> provide
>>> simplified, Pythonic interface, but that is not directly relevant to
>>> my question here).
>>>
>>> I'm trying to keep the code as simple & generic as possible, so my
>>> question is
>>> not about Python but unit testing and mocking.
>>>
>>> # xdep.py
>>> def open():
>>> ...
>>> def get_item(item_id):
>>> ...
>>> def close():
>>> ...
>>>
>>> # foo.py
>>> import xdep
>>>
>>> def exists(item_id):
>>> xdep.open()
>>> item = xdep.get_item(item_id)
>>> xdep.close()
>>> if item:
>>> return item.id == item_id
>>> else:
>>> return false
>>>
>>> Here is my module to unit test behaviuour of the foo:
>>> - I create and plug a stub for the legacy module xdep
>>> - I mock functions of xdep
>>>
>>> # test_foo.py
>>> import unittest
>>> import unittest.mock as mock
>>>
>>> class ItemTest(unittest.TestCase):
>>> def setUp(self):
>>>
>>> # create stub for external dependency of foo
>>> self.xdep_stub = mock.Mock()
>>> patcher = mock.patch.dict('sys.modules', {'xdep': self.xdep_stub})
>>> self.addCleanup(patcher.stop)
>>> patcher.start()
>>>
>>> # import foo does import xdep, as patched above
>>> import foo
>>> self.foo = foo
>>>
>>> def test_exists_ExistingItemId_ReturnTrue(self):
>>> foo = self.foo
>>> # Arrange
>>> ### Data
>>> item_id = 1
>>> ### Mock
>>> foo.xdep.get_item.return_value = 1
>>> # Act
>>> item_exists = foo.item.exists(item_id)
>>> # Assert
>>> self.assertTrue(item_exists)
>>>
>>> I have a single assertion and I use single mock which mocks xdep.get_item
>>> function, so it seems a valid yet clear unit test.
>>>
>>> Now, having the single assert/single mock rule in mind, I have some
>>> questions,
>>> about the test_exists_ExistingItemId_ReturnTrue in particular:
>>>
>>> 1) How asserting on foo.item.exists return value is different from
>>> asserting how xdep.get_item mock was called?
>>>
>>> I could have replaced
>>> self.assertTrue(item_exists)
>>> with
>>> foo.xdep.get_item.assert_called_once_with(1)
>>>
>>> 2) What about having both assertions, would that break the single
>>> assertion rule?
>>>
>>> self.assertTrue(item_exists)
>>> foo.xdep.get_item.assert_called_once_with(1)
>>>
>>> I'm not concern about breaking the rule as technical convention, but as:
>>> am I testing multiple things here?
>>>
>>> 3) AFAIU, it's natural that unit tests are coupled with implementation of
>>> method/behaviour they are testing, foo.exists function in my case.
>>>
>>> So, given that foo.exists function calls three functions of the legacy
>>> xdep module
>>> xdep.open
>>> xdep.get_item
>>> xdep.close
>>> should I mock all the three function and verify how they were called?
>>> Would that still belong to scope of
>>> test_exists_ExistingItemId_ReturnTrue test
>>> or I better create a new dedicated test case(s), something like
>>> test_exists_IfCalled_xdepOpenCalled
>>> test_exists_IfCalled_xdepCloseCalled
>>> test_exists_IfCalled_xdepGetItemCalled
>>> and do assert_called_once_with() appropriately.
>>>
>>> Above, I count each mock for each xdep function as a separate mock,
>>> so "single mock per test" rule seems to suggests I should verify they
>>> are called
>>> in separate tests, doesn't it?
>>>
>>> Also, some of other xdep functions need to be called with multiple
>>> parameters,
>>> and as setting up and verifying their mocks may be quite elaborate,my
>>> gut feeling
>>> tells me that single unit test per "legacy API call expectation" as
>>> listed above
>>> is a good approach for readability and maintainability of tests.
>>> Am I right?
>>>
>>> 4) What other, if any, approach to single assert/single mock rules
>>> would you take for testing similar case as above?
>>>
>>> Best regards,
>>> --
>>> Mateusz Łoskot, http://mateusz.loskot.net
>>>
>>
>>
>  
>



-- 
Thanks,

Roy Osherove

   - *@RoyOsherove* <https://twitter.com/RoyOsherove>
   - Read my new book *Notes to a Software Team Leader
<http://TeamLeadSkills.com>*
   - My blog for team leaders: *http://5Whys.com* <http://5Whys.com>
   - +47-96-90-22-15 (Oslo Time)