3

I'm new to unit testing and want to start with the nose framework. But answers using unittest, pytest are welcome too. Also of course, general advice.

I think I get the basic concept but I lack practice in setting up good test. I'm also struggling with how to layout the tests. I'm especially unsure on how to approach the case where I want to run a couple of test cases on different function of a module:

For example: I might have a module called diceroller.py it contains a couple of function on simulating rolling dice, modify and test the results and so on. All functions for rolling dice should run trough the same tests (do they return a list of integers, with the right amount of values, are the values in range, etc.). But some of those should also be run against some additional cases.

So I got a subdir test and want to setup my testcode there. How do I approach this?

# a section of `diceroller.py` def roll_a_dice(sides=6): """Return an integer x where `x >= 1 <= sides`""" return random.randint(1, sides) def roll_dice(sides=6, count=1): """Return a list of integers (most function except this)""" rolls = list() while count: rolls.append(random.randint(1, sides)) count -= 1 return rolls def roll_some_dice(sides=6, count=1, times=1): """Return a list of list containing integers""" some_rolls = list() while times: some_rolls.append(roll_dice(sides, count)) times -= 1 return some_rolls def rolling_dice(sides=6, count=1): """Yielding integers `count` times""" while count: count -= 1 yield random.randint(1, sides)

Small update

Simeon Visser has a good point. But the code above where just functions too ad some context to my Questions, which just is: How can I (re)use test cases on different functions?

I guess writing tests like check_xyz and then calling it from test_a and test_b for example, is the easiest solution? Or is this bad practice?

The solution from Rik Poggi seems to do exactly what is was trying to accomplish (will play around with it right after typing this). But I kinda have the feeling it "complicates" things to much... probably not on the technical side but it comes as "too much" maybe.

Improve this question
1
  • 1
    One great thing about testing is that it helps to reveal where you should refactor. Before thinking about trying to reuse tests, I would think about factoring out the code common to dice rolling, and testing that. For example, all of your functions above are just special cases of role_some_dice. Commented Mar 22, 2012 at 16:16

2 Answers 2

Reset to default
9

I will not address the problem that your sample code to test clearly have. I'll focus on your question about reusing test code.

Important things to keep in mind before even starting:

  1. Tests must be simple (without any logic inside, or very very little).
  2. Tests should be readable (they are like code documentation).
  3. Tests should be maintainable.

The simplicity of your tests is a must, and while keeping that you'll need to find the right balance between the last two points.

One last warning: Be very very careful when reusing your test code, because if you'll do it wrong your tests will be bugged. Test bugs are hard to spot, but a year from now you may find one, the seed of doubt will be planted and the trust in your tests might decrease. Tests that you don't trust ("oh, that one is failing but nevermind because that other one is failing too, etc...") are completely and utterly worthless.

Now that we cleared our way let's look at your code. Usually the dynamic part of tests is achieved through context (the setUp and tearDown methods), but in your case things are a little more complex.

I want to run a couple of test cases on different function of a module.

You don't actually want the same test case to run with a different function, but only the same code. A good testing environment will have (at least) one test case for every function.

Since you're looking for the ability to run a previous test case/suite against a partial output of another function, you'll need functools.partial that will let you wrap your function with default arguments.

This means that you should start from the bottom, from the simplest tests:

def check_valid_range(value, sides): """Check that value is a valid dice rolling""" assert 0 < value <= sides def check_is_int(value): """Check that value is an integer""" assert type(value) is int

And then build on top of them (with a little pluggability):

class TestRollADice: """roll_a_dice basic tests""" @staticmethod def func_to_test(*args, **kwargs): return diceroller.roll_a_dice(*args, **kwargs) @staticmethod def check_valid_output(value, sides): """Check that value is the self.function is valid""" yield check_valid_range, value, sides yield check_is_int, value def test_3sides(self): """Check valid result for a 3 sides dice""" yield self.check_valid_output, self.func_to_test(3), 3 def test_list_valid_sides(self): """Check valid result for a list of valid sides (sides >= 3)""" sides = list(range(3, 13)) for s in sides: yield self.check_valid_output, self.func_to_test(s), s def test_0_sides_raises_ValueError(self): """0 side dice raise ValueError""" assert_raises(ValueError, self.func_to_test, 0) def test_1_sides_raises_ValueError(self): """1 side dice raise ValueError""" assert_raises(ValueError, self.func_to_test, 1) def test_2_sides_raises_ValueError(self): """2 sides dice raise ValueError""" assert_raises(ValueError, self.func_to_test, 2) def test_minus1_sides_raises_ValueError(self): """-1 side dice raise ValueError""" assert_raises(ValueError, self.func_to_test, -1)

These are the minimum amount of tests your roll_a_dice function should have. Try to run nosetests in your package and you'll see two of them failing, see how tests are important!! :)

I'm sure you've noticed how the tests name are verbose, it's for readability.

Now for the roll_dice the main test class should test those three, four basic values and the baisc errors, because a good should be simple and keep under control what's being tested, I'd say that you might need some functions like:

def check_list_length(self, lst, count): """Check that the len(lst) == count""" assert len(lst) == count def check_valid_count(self, count): """Check count > 0""" assert count > 0 class TestRollDice: # ...

And now if you want to reuse the old code you can subclass TestRollADice:

from functools import partial class TestRollDice_Count1(TestRollADice): @staticmethod def func_to_test(*args, **kwargs): return partial(diceroller.roll_dice, count=1)(*args, **kwargs)

And voilà, almost free you'll have twice the tests as before :)

Note:

  1. All the code above could be written in unittest language, but since you asked about nose I wrote with that.
  2. There's a minor problem: the docstrings for the passing tests are the same, so in the verbose output they show up twice. It's not a big problem, since if one of those tests should fail you'll be prompted with the test address and that one is unique (and clear since luckly our test names are verbose). I also seem to remember that the docstrings might be disabled or something. So, once again, this is not a big problem, and should be easy to find a workaround, if needed.
Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

Thanks. This was exactly what I was looking for. But can't I just code check_x functions (to check for int, check for right length, etc.) and then call those from different test_x functions or classes (test_roll_a_dice, test_roll_dice and so on)?
@Brutus: I'd say no.. the point is that good unit tests aren't build that way, a good that function shoult assert one thing and one thing only. There are obviously rare case where you need multiple asserts, and your example is one of them (how lucky eh?). The "problem" is that the roll_dice functions return random numbers, so to check if the output was valid (in range and integer) I had to put two assertion inside the check_valid_output. Usually you wouldn't have this last one aggregating check. So you usually just a have a list of test_*** functions each one testing one thing.
[continue] The point is that tests are grouped under test suites and not under functions, so you should try to reuse those suites. Context managing (tearDown, setUp, ContextSuite, etc...) should help you with that. Have I answered your question?
@RikPoggi - I love these kind of answers... +1 for having taken the time to explain the ground of test design. :)
4

Your code can be refactored to reduce the amount of code and therefore the amount of code that you need to test. Let's look at what you want:

  • Roll a dice (roll_a_dice)
  • Roll a dice a number of times (roll_dice)
  • Get a number of rolls in which a dice has been rolled a number of times (roll_some_dice)

Let's rewrite roll_dice first. We basically want to call roll_a_dice a number of times and put the results in a list:

def roll_dice(sides=6, count=1): """Return a list of integers (most function except this)""" return [roll_a_dice(sides) for i in xrange(count)]

Let's do the same thing with roll_some_dice: we want to call roll_dice a number of times and put the results in a list:

def roll_some_dice(sides=6, count=1, times=1): """Return a list of list containing integers""" return [roll_dice(sides, count) for i in xrange(times)]

Lastly, rolling_dice still contains some logic:

def rolling_dice(sides=6, count=1): """Yielding integers `count` times""" while count: count -= 1 yield roll_a_dice(sides)

The code is now a lot easier to check for errors and easier to unit test:

import random def roll_a_dice(sides=6): """Return an integer x where `x >= 1 <= sides`""" return random.randint(1, sides) def roll_dice(sides=6, count=1): """Return a list of integers (most function except this)""" return [roll_a_dice(sides) for i in xrange(count)] def roll_some_dice(sides=6, count=1, times=1): """Return a list of list containing integers""" return [roll_dice(sides, count) for i in xrange(times)] def rolling_dice(sides=6, count=1): """Yielding integers `count` times""" while count: count -= 1 yield roll_a_dice(sides)

You can now write a test case to check whether rolling a dice goes correctly (roll_a_dice). Once you've tested that, you don't need to do that anymore for the other functions. For those functions, you only need to test whether the correct number of results is obtained (because any incorrect values there should have been caught by the test case(s) for roll_a_dice).

Similarly, when you're testing roll_some_dice you can assume roll_dice works correctly because you've already written tests for that function. If something does go wrong, you can add a test to the test case for roll_dice, not roll_some_dice (unless the problem is specific for roll_some_dice).

Improve this answer

2 Comments

+1. Thanks for fleshing the ideas of my comment out into a full answer. Also, the use of list comprehension makes this code much more readable imo. I want to point out again that testing not only ensures your code is correct, but helps inform better design, as we can see here.
Great answer and a good explanation. But I was just making that functions up as a (I admit - bad) example ;)

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.