1 ==========================
2 testtools for test authors
3 ==========================
5 If you are writing tests for a Python project and you (rather wisely) want to
6 use testtools to do so, this is the manual for you.
8 We assume that you already know Python and that you know something about
9 automated testing already.
11 If you are a test author of an unusually large or unusually unusual test
12 suite, you might be interested in :doc:`for-framework-folk`.
14 You might also be interested in the `testtools API docs`_.
20 testtools is a set of extensions to Python's standard unittest module.
21 Writing tests with testtools is very much like writing tests with standard
22 Python, or with Twisted's "trial_", or nose_, except a little bit easier and
25 Below, we'll try to give some examples of how to use testtools in its most
26 basic way, as well as a sort of feature-by-feature breakdown of the cool bits
27 that you could easily miss.
33 Here's what a basic testtools unit tests look like::
35 from testtools import TestCase
36 from myproject import silly
38 class TestSillySquare(TestCase):
39 """Tests for silly square function."""
41 def test_square(self):
42 # 'square' takes a number and multiplies it by itself.
43 result = silly.square(7)
44 self.assertEqual(result, 49)
46 def test_square_bad_input(self):
47 # 'square' raises a TypeError if it's given bad input, say a
49 self.assertRaises(TypeError, silly.square, "orange")
52 Here you have a class that inherits from ``testtools.TestCase`` and bundles
53 together a bunch of related tests. The tests themselves are methods on that
54 class that begin with ``test_``.
59 You can run these tests in many ways. testtools provides a very basic
60 mechanism for doing so::
62 $ python -m testtools.run exampletest
68 where 'exampletest' is a module that contains unit tests. By default,
69 ``testtools.run`` will *not* recursively search the module or package for unit
70 tests. To do this, you will need to either have the discover_ module
71 installed or have Python 2.7 or later, and then run::
73 $ python -m testtools.run discover packagecontainingtests
75 For more information see the Python 2.7 unittest documentation, or::
77 python -m testtools.run --help
79 As your testing needs grow and evolve, you will probably want to use a more
80 sophisticated test runner. There are many of these for Python, and almost all
81 of them will happily run testtools tests. In particular:
87 * `zope.testrunner`_ (aka zope.testing)
89 From now on, we'll assume that you know how to run your tests.
91 Running test with Distutils
92 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
94 If you are using Distutils_ to build your Python project, you can use the testtools
95 Distutils_ command to integrate testtools into your Distutils_ workflow::
97 from distutils.core import setup
98 from testtools import TestCommand
102 cmdclass={'test': TestCommand}
107 $ python setup.py test -m exampletest
109 Ran 2 tests in 0.000s
113 For more information about the capabilities of the `TestCommand` command see::
115 $ python setup.py test --help
117 You can use the `setup configuration`_ to specify the default behavior of the
118 `TestCommand` command.
123 The core of automated testing is making assertions about the way things are,
124 and getting a nice, helpful, informative error message when things are not as
127 All of the assertions that you can find in Python standard unittest_ can be
128 found in testtools (remember, testtools extends unittest). testtools changes
129 the behaviour of some of those assertions slightly and adds some new
130 assertions that you will almost certainly find useful.
133 Improved assertRaises
134 ---------------------
136 ``TestCase.assertRaises`` returns the caught exception. This is useful for
137 asserting more things about the exception than just the type::
139 def test_square_bad_input(self):
140 # 'square' raises a TypeError if it's given bad input, say a
142 e = self.assertRaises(TypeError, silly.square, "orange")
143 self.assertEqual("orange", e.bad_value)
144 self.assertEqual("Cannot square 'orange', not a number.", str(e))
146 Note that this is incompatible with the ``assertRaises`` in unittest2 and
153 If you are using a version of Python that supports the ``with`` context
154 manager syntax, you might prefer to use that syntax to ensure that code raises
155 particular errors. ``ExpectedException`` does just that. For example::
157 def test_square_root_bad_input_2(self):
158 # 'square' raises a TypeError if it's given bad input.
159 with ExpectedException(TypeError, "Cannot square.*"):
160 silly.square('orange')
162 The first argument to ``ExpectedException`` is the type of exception you
163 expect to see raised. The second argument is optional, and can be either a
164 regular expression or a matcher. If it is a regular expression, the ``str()``
165 of the raised exception must match the regular expression. If it is a matcher,
166 then the raised exception object must match it.
169 assertIn, assertNotIn
170 ---------------------
172 These two assertions check whether a value is in a sequence and whether a
173 value is not in a sequence. They are "assert" versions of the ``in`` and
174 ``not in`` operators. For example::
176 def test_assert_in_example(self):
177 self.assertIn('a', 'cat')
178 self.assertNotIn('o', 'cat')
179 self.assertIn(5, list_of_primes_under_ten)
180 self.assertNotIn(12, list_of_primes_under_ten)
183 assertIs, assertIsNot
184 ---------------------
186 These two assertions check whether values are identical to one another. This
187 is sometimes useful when you want to test something more strict than mere
188 equality. For example::
190 def test_assert_is_example(self):
194 self.assertIs(foo, foo_alias)
195 self.assertIsNot(foo, bar)
196 self.assertEqual(foo, bar) # They are equal, but not identical
202 As much as we love duck-typing and polymorphism, sometimes you need to check
203 whether or not a value is of a given type. This method does that. For
206 def test_assert_is_instance_example(self):
208 self.assertIsInstance(now, datetime)
210 Note that there is no ``assertIsNotInstance`` in testtools currently.
216 Sometimes it's useful to write tests that fail. For example, you might want
217 to turn a bug report into a unit test, but you don't know how to fix the bug
218 yet. Or perhaps you want to document a known, temporary deficiency in a
221 testtools gives you the ``TestCase.expectFailure`` to help with this. You use
222 it to say that you expect this assertion to fail. When the test runs and the
223 assertion fails, testtools will report it as an "expected failure".
227 def test_expect_failure_example(self):
229 "cats should be dogs", self.assertEqual, 'cats', 'dogs')
231 As long as 'cats' is not equal to 'dogs', the test will be reported as an
234 If ever by some miracle 'cats' becomes 'dogs', then testtools will report an
235 "unexpected success". Unlike standard unittest, testtools treats this as
236 something that fails the test suite, like an error or a failure.
242 The built-in assertion methods are very useful, they are the bread and butter
243 of writing tests. However, soon enough you will probably want to write your
244 own assertions. Perhaps there are domain specific things that you want to
245 check (e.g. assert that two widgets are aligned parallel to the flux grid), or
246 perhaps you want to check something that could almost but not quite be found
247 in some other standard library (e.g. assert that two paths point to the same
250 When you are in such situations, you could either make a base class for your
251 project that inherits from ``testtools.TestCase`` and make sure that all of
252 your tests derive from that, *or* you could use the testtools ``Matcher``
259 Here's a really basic example using stock matchers found in testtools::
262 from testtools.matchers import Equals
264 class TestSquare(TestCase):
265 def test_square(self):
267 self.assertThat(result, Equals(49))
269 The line ``self.assertThat(result, Equals(49))`` is equivalent to
270 ``self.assertEqual(result, 49)`` and means "assert that ``result`` equals 49".
271 The difference is that ``assertThat`` is a more general method that takes some
272 kind of observed value (in this case, ``result``) and any matcher object
273 (here, ``Equals(49)``).
275 The matcher object could be absolutely anything that implements the Matcher
276 protocol. This means that you can make more complex matchers by combining
279 def test_square_silly(self):
281 self.assertThat(result, Not(Equals(50)))
283 Which is roughly equivalent to::
285 def test_square_silly(self):
287 self.assertNotEqual(result, 50)
293 testtools comes with many matchers built in. They can all be found in and
294 imported from the ``testtools.matchers`` module.
299 Matches if two items are equal. For example::
301 def test_equals_example(self):
302 self.assertThat([42], Equals([42]))
308 Matches if two items are identical. For example::
310 def test_is_example(self):
312 self.assertThat(foo, Is(foo))
318 Adapts isinstance() to use as a matcher. For example::
320 def test_isinstance_example(self):
322 self.assertThat(MyClass(), IsInstance(MyClass))
323 self.assertThat(MyClass(), IsInstance(MyClass, str))
329 Matches if a callable raises a particular type of exception. For example::
331 def test_raises_example(self):
332 self.assertThat(lambda: 1/0, raises(ZeroDivisionError))
334 This is actually a convenience function that combines two other matchers:
335 Raises_ and MatchesException_.
341 Matches a string as if it were the output of a doctest_ example. Very useful
342 for making assertions about large chunks of text. For example::
346 def test_doctest_example(self):
347 output = "Colorless green ideas"
350 DocTestMatches("Colorless ... ideas", doctest.ELLIPSIS))
352 We highly recommend using the following flags::
354 doctest.ELLIPSIS | doctest.NORMALIZE_WHITESPACE | doctest.REPORT_NDIFF
360 Matches if the given thing is greater than the thing in the matcher. For
363 def test_greater_than_example(self):
364 self.assertThat(3, GreaterThan(2))
370 Matches if the given thing is less than the thing in the matcher. For
373 def test_less_than_example(self):
374 self.assertThat(2, LessThan(3))
380 These matchers check to see if a string starts with or ends with a particular
381 substring. For example::
383 def test_starts_and_ends_with_example(self):
384 self.assertThat('underground', StartsWith('und'))
385 self.assertThat('underground', EndsWith('und'))
391 This matcher checks to see if the given thing contains the thing in the
392 matcher. For example::
394 def test_contains_example(self):
395 self.assertThat('abc', Contains('b'))
401 Matches an exc_info tuple if the exception is of the correct type. For
404 def test_matches_exception_example(self):
406 raise RuntimeError('foo')
408 exc_info = sys.exc_info()
409 self.assertThat(exc_info, MatchesException(RuntimeError))
410 self.assertThat(exc_info, MatchesException(RuntimeError('bar'))
412 Most of the time, you will want to uses `The raises helper`_ instead.
418 Matches if something is not equal to something else. Note that this is subtly
419 different to ``Not(Equals(x))``. ``NotEquals(x)`` will match if ``y != x``,
420 ``Not(Equals(x))`` will match if ``not y == x``.
422 You only need to worry about this distinction if you are testing code that
423 relies on badly written overloaded equality operators.
429 Matches if the keys of one dict are equal to the keys of another dict. For
432 def test_keys_equal(self):
435 self.assertThat(a, KeysEqual(b))
441 Matches a string against a regular expression, which is a wonderful thing to
442 be able to do, if you think about it::
444 def test_matches_regex_example(self):
445 self.assertThat('foo', MatchesRegex('fo+'))
451 One great thing about matchers is that you can readily combine existing
452 matchers to get variations on their behaviour or to quickly build more complex
455 Below are a few of the combining matchers that come with testtools.
461 Negates another matcher. For example::
463 def test_not_example(self):
464 self.assertThat([42], Not(Equals("potato")))
465 self.assertThat([42], Not(Is([42])))
467 If you find yourself using ``Not`` frequently, you may wish to create a custom
468 matcher for it. For example::
470 IsNot = lambda x: Not(Is(x))
472 def test_not_example_2(self):
473 self.assertThat([42], IsNot([42]))
479 Used to add custom notes to a matcher. For example::
481 def test_annotate_example(self):
484 result, Annotate("Not the answer to the Question!", Equals(42))
486 Since the annotation is only ever displayed when there is a mismatch
487 (e.g. when ``result`` does not equal 42), it's a good idea to phrase the note
488 negatively, so that it describes what a mismatch actually means.
490 As with Not_, you may wish to create a custom matcher that describes a
491 common operation. For example::
493 PoliticallyEquals = lambda x: Annotate("Death to the aristos!", Equals(x))
495 def test_annotate_example_2(self):
496 self.assertThat("orange", PoliticallyEquals("yellow"))
498 You can have assertThat perform the annotation for you as a convenience::
500 def test_annotate_example_3(self):
501 self.assertThat("orange", Equals("yellow"), "Death to the aristos!")
507 Used to make a matcher that applies a function to the matched object before
508 matching. This can be used to aid in creating trivial matchers as functions, for
511 def test_after_preprocessing_example(self):
512 def HasFileContent(content):
514 return open(path).read()
515 return AfterPreprocessing(_read, Equals(content))
516 self.assertThat('/tmp/foo.txt', PathHasFileContent("Hello world!"))
522 Combines many matchers to make a new matcher. The new matcher will only match
523 things that match every single one of the component matchers.
525 It's much easier to understand in Python than in English::
527 def test_matches_all_example(self):
528 has_und_at_both_ends = MatchesAll(StartsWith("und"), EndsWith("und"))
530 self.assertThat("underground", has_und_at_both_ends)
532 self.assertThat("found", has_und_at_both_ends)
534 self.assertThat("undead", has_und_at_both_ends)
536 At this point some people ask themselves, "why bother doing this at all? why
537 not just have two separate assertions?". It's a good question.
539 The first reason is that when a ``MatchesAll`` gets a mismatch, the error will
540 include information about all of the bits that mismatched. When you have two
541 separate assertions, as below::
543 def test_two_separate_assertions(self):
544 self.assertThat("foo", StartsWith("und"))
545 self.assertThat("foo", EndsWith("und"))
547 Then you get absolutely no information from the second assertion if the first
548 assertion fails. Tests are largely there to help you debug code, so having
549 more information in error messages is a big help.
551 The second reason is that it is sometimes useful to give a name to a set of
552 matchers. ``has_und_at_both_ends`` is a bit contrived, of course, but it is
559 Like MatchesAll_, ``MatchesAny`` combines many matchers to make a new
560 matcher. The difference is that the new matchers will match a thing if it
561 matches *any* of the component matchers.
565 def test_matches_any_example(self):
566 self.assertThat(42, MatchesAny(Equals(5), Not(Equals(6))))
572 Matches many values against a single matcher. Can be used to make sure that
573 many things all meet the same condition::
575 def test_all_match_example(self):
576 self.assertThat([2, 3, 5, 7], AllMatch(LessThan(10)))
578 If the match fails, then all of the values that fail to match will be included
579 in the error message.
581 In some ways, this is the converse of MatchesAll_.
587 Where ``MatchesAny`` and ``MatchesAll`` combine many matchers to match a
588 single value, ``MatchesListwise`` combines many matches to match many values.
592 def test_matches_listwise_example(self):
594 [1, 2, 3], MatchesListwise(map(Equals, [1, 2, 3])))
596 This is useful for writing custom, domain-specific matchers.
602 Combines many matchers to match many values, without regard to their order.
606 def test_matches_setwise_example(self):
608 [1, 2, 3], MatchesSetwise(Equals(2), Equals(3), Equals(1)))
610 Much like ``MatchesListwise``, best used for writing custom, domain-specific
617 Creates a matcher that matches certain attributes of an object against a
618 pre-defined set of matchers.
620 It's much easier to understand in Python than in English::
622 def test_matches_structure_example(self):
626 matcher = MatchesStructure(a=Equals(1), b=Equals(2))
627 self.assertThat(foo, matcher)
629 Since all of the matchers used were ``Equals``, we could also write this using
630 the ``byEquality`` helper::
632 def test_matches_structure_example(self):
636 matcher = MatchesStructure.byEquality(a=1, b=2)
637 self.assertThat(foo, matcher)
639 ``MatchesStructure.fromExample`` takes an object and a list of attributes and
640 creates a ``MatchesStructure`` matcher where each attribute of the matched
641 object must equal each attribute of the example object. For example::
643 matcher = MatchesStructure.fromExample(foo, 'a', 'b')
645 is exactly equivalent to ``matcher`` in the previous example.
651 Takes whatever the callable raises as an exc_info tuple and matches it against
652 whatever matcher it was given. For example, if you want to assert that a
653 callable raises an exception of a given type::
655 def test_raises_example(self):
657 lambda: 1/0, Raises(MatchesException(ZeroDivisionError)))
659 Although note that this could also be written as::
661 def test_raises_example_convenient(self):
662 self.assertThat(lambda: 1/0, raises(ZeroDivisionError))
664 See also MatchesException_ and `the raises helper`_
667 Writing your own matchers
668 -------------------------
670 Combining matchers is fun and can get you a very long way indeed, but
671 sometimes you will have to write your own. Here's how.
673 You need to make two closely-linked objects: a ``Matcher`` and a
674 ``Mismatch``. The ``Matcher`` knows how to actually make the comparison, and
675 the ``Mismatch`` knows how to describe a failure to match.
677 Here's an example matcher::
679 class IsDivisibleBy(object):
680 """Match if a number is divisible by another number."""
681 def __init__(self, divider):
682 self.divider = divider
684 return 'IsDivisibleBy(%s)' % (self.divider,)
685 def match(self, actual):
686 remainder = actual % self.divider
688 return IsDivisibleByMismatch(actual, self.divider, remainder)
692 The matcher has a constructor that takes parameters that describe what you
693 actually *expect*, in this case a number that other numbers ought to be
694 divisible by. It has a ``__str__`` method, the result of which is displayed
695 on failure by ``assertThat`` and a ``match`` method that does the actual
698 ``match`` takes something to match against, here ``actual``, and decides
699 whether or not it matches. If it does match, then ``match`` must return
700 ``None``. If it does *not* match, then ``match`` must return a ``Mismatch``
701 object. ``assertThat`` will call ``match`` and then fail the test if it
702 returns a non-None value. For example::
704 def test_is_divisible_by_example(self):
705 # This succeeds, since IsDivisibleBy(5).match(10) returns None.
706 self.assertThat(10, IsDivisbleBy(5))
707 # This fails, since IsDivisibleBy(7).match(10) returns a mismatch.
708 self.assertThat(10, IsDivisbleBy(7))
710 The mismatch is responsible for what sort of error message the failing test
711 generates. Here's an example mismatch::
713 class IsDivisibleByMismatch(object):
714 def __init__(self, number, divider, remainder):
716 self.divider = divider
717 self.remainder = remainder
720 return "%r is not divisible by %r, %r remains" % (
721 self.number, self.divider, self.remainder)
723 def get_details(self):
726 The mismatch takes information about the mismatch, and provides a ``describe``
727 method that assembles all of that into a nice error message for end users.
728 You can use the ``get_details`` method to provide extra, arbitrary data with
729 the mismatch (e.g. the contents of a log file). Most of the time it's fine to
730 just return an empty dict. You can read more about Details_ elsewhere in this
733 Sometimes you don't need to create a custom mismatch class. In particular, if
734 you don't care *when* the description is calculated, then you can just do that
735 in the Matcher itself like this::
737 def match(self, actual):
738 remainder = actual % self.divider
741 "%r is not divisible by %r, %r remains" % (
742 actual, self.divider, remainder))
746 When writing a ``describe`` method or constructing a ``Mismatch`` object the
747 code should ensure it only emits printable unicode. As this output must be
748 combined with other text and forwarded for presentation, letting through
749 non-ascii bytes of ambiguous encoding or control characters could throw an
750 exception or mangle the display. In most cases simply avoiding the ``%s``
751 format specifier and using ``%r`` instead will be enough. For examples of
752 more complex formatting see the ``testtools.matchers`` implementatons.
758 As we may have mentioned once or twice already, one of the great benefits of
759 automated tests is that they help find, isolate and debug errors in your
762 Frequently however, the information provided by a mere assertion failure is
763 not enough. It's often useful to have other information: the contents of log
764 files; what queries were run; benchmark timing information; what state certain
765 subsystem components are in and so forth.
767 testtools calls all of these things "details" and provides a single, powerful
768 mechanism for including this information in your test run.
770 Here's an example of how to add them::
772 from testtools import TestCase
773 from testtools.content import text_content
775 class TestSomething(TestCase):
777 def test_thingy(self):
778 self.addDetail('arbitrary-color-name', text_content("blue"))
779 1 / 0 # Gratuitous error!
781 A detail an arbitrary piece of content given a name that's unique within the
782 test. Here the name is ``arbitrary-color-name`` and the content is
783 ``text_content("blue")``. The name can be any text string, and the content
784 can be any ``testtools.content.Content`` object.
786 When the test runs, testtools will show you something like this::
788 ======================================================================
789 ERROR: exampletest.TestSomething.test_thingy
790 ----------------------------------------------------------------------
791 arbitrary-color-name: {{{blue}}}
793 Traceback (most recent call last):
794 File "exampletest.py", line 8, in test_thingy
795 1 / 0 # Gratuitous error!
796 ZeroDivisionError: integer division or modulo by zero
800 As you can see, the detail is included as an attachment, here saying
801 that our arbitrary-color-name is "blue".
807 For the actual content of details, testtools uses its own MIME-based Content
808 object. This allows you to attach any information that you could possibly
809 conceive of to a test, and allows testtools to use or serialize that
812 The basic ``testtools.content.Content`` object is constructed from a
813 ``testtools.content.ContentType`` and a nullary callable that must return an
814 iterator of chunks of bytes that the content is made from.
816 So, to make a Content object that is just a simple string of text, you can
819 from testtools.content import Content
820 from testtools.content_type import ContentType
822 text = Content(ContentType('text', 'plain'), lambda: ["some text"])
824 Because adding small bits of text content is very common, there's also a
827 text = text_content("some text")
829 To make content out of an image stored on disk, you could do something like::
831 image = Content(ContentType('image', 'png'), lambda: open('foo.png').read())
833 Or you could use the convenience function::
835 image = content_from_file('foo.png', ContentType('image', 'png'))
837 The ``lambda`` helps make sure that the file is opened and the actual bytes
838 read only when they are needed – by default, when the test is finished. This
839 means that tests can construct and add Content objects freely without worrying
840 too much about how they affect run time.
846 A very common use of details is to add a log file to failing tests. Say your
847 project has a server represented by a class ``SomeServer`` that you can start
848 up and shut down in tests, but runs in another process. You want to test
849 interaction with that server, and whenever the interaction fails, you want to
850 see the client-side error *and* the logs from the server-side. Here's how you
853 from testtools import TestCase
854 from testtools.content import attach_file, Content
855 from testtools.content_type import UTF8_TEXT
857 from myproject import SomeServer
859 class SomeTestCase(TestCase):
862 super(SomeTestCase, self).setUp()
863 self.server = SomeServer()
864 self.server.start_up()
865 self.addCleanup(self.server.shut_down)
866 self.addCleanup(attach_file, self.server.logfile, self)
868 def attach_log_file(self):
872 lambda: open(self.server.logfile, 'r').readlines()))
874 def test_a_thing(self):
875 self.assertEqual("cool", self.server.temperature)
877 This test will attach the log file of ``SomeServer`` to each test that is
878 run. testtools will only display the log file for failing tests, so it's not
881 If the act of adding at detail is expensive, you might want to use
882 addOnException_ so that you only do it when a test actually raises an
886 Controlling test execution
887 ==========================
894 ``TestCase.addCleanup`` is a robust way to arrange for a clean up function to
895 be called before ``tearDown``. This is a powerful and simple alternative to
896 putting clean up logic in a try/finally block or ``tearDown`` method. For
901 self.addCleanup(foo.unlock)
904 This is particularly useful if you have some sort of factory in your test::
906 def make_locked_foo(self):
909 self.addCleanup(foo.unlock)
912 def test_frotz_a_foo(self):
913 foo = self.make_locked_foo()
915 self.assertEqual(foo.frotz_count, 1)
917 Any extra arguments or keyword arguments passed to ``addCleanup`` are passed
918 to the callable at cleanup time.
920 Cleanups can also report multiple errors, if appropriate by wrapping them in
921 a ``testtools.MultipleExceptions`` object::
923 raise MultipleExceptions(exc_info1, exc_info2)
929 Tests often depend on a system being set up in a certain way, or having
930 certain resources available to them. Perhaps a test needs a connection to the
931 database or access to a running external server.
933 One common way of doing this is to do::
935 class SomeTest(TestCase):
937 super(SomeTest, self).setUp()
938 self.server = Server()
940 self.addCleanup(self.server.tearDown)
942 testtools provides a more convenient, declarative way to do the same thing::
944 class SomeTest(TestCase):
946 super(SomeTest, self).setUp()
947 self.server = self.useFixture(Server())
949 ``useFixture(fixture)`` calls ``setUp`` on the fixture, schedules a clean up
950 to clean it up, and schedules a clean up to attach all details_ held by the
951 fixture to the test case. The fixture object must meet the
952 ``fixtures.Fixture`` protocol (version 0.3.4 or newer, see fixtures_).
954 If you have anything beyond the most simple test set up, we recommend that
955 you put this set up into a ``Fixture`` class. Once there, the fixture can be
956 easily re-used by other tests and can be combined with other fixtures to make
957 more complex resources.
963 Many reasons exist to skip a test: a dependency might be missing; a test might
964 be too expensive and thus should not berun while on battery power; or perhaps
965 the test is testing an incomplete feature.
967 ``TestCase.skipTest`` is a simple way to have a test stop running and be
968 reported as a skipped test, rather than a success, error or failure. For
971 def test_make_symlink(self):
972 symlink = getattr(os, 'symlink', None)
974 self.skipTest("No symlink support")
975 symlink(whatever, something_else)
977 Using ``skipTest`` means that you can make decisions about what tests to run
978 as late as possible, and close to the actual tests. Without it, you might be
979 forced to use convoluted logic during test loading, which is a bit of a mess.
985 If you are using this feature when running your test suite with a legacy
986 ``TestResult`` object that is missing the ``addSkip`` method, then the
987 ``addError`` method will be invoked instead. If you are using a test result
988 from testtools, you do not have to worry about this.
990 In older versions of testtools, ``skipTest`` was known as ``skip``. Since
991 Python 2.7 added ``skipTest`` support, the ``skip`` name is now deprecated.
992 No warning is emitted yet – some time in the future we may do so.
998 Sometimes, you might wish to do something only when a test fails. Perhaps you
999 need to run expensive diagnostic routines or some such.
1000 ``TestCase.addOnException`` allows you to easily do just this. For example::
1002 class SomeTest(TestCase):
1004 super(SomeTest, self).setUp()
1005 self.server = self.useFixture(SomeServer())
1006 self.addOnException(self.attach_server_diagnostics)
1008 def attach_server_diagnostics(self, exc_info):
1009 self.server.prep_for_diagnostics() # Expensive!
1010 self.addDetail('server-diagnostics', self.server.get_diagnostics)
1012 def test_a_thing(self):
1013 self.assertEqual('cheese', 'chalk')
1015 In this example, ``attach_server_diagnostics`` will only be called when a test
1016 fails. It is given the exc_info tuple of the error raised by the test, just
1017 in case it is needed.
1023 testtools provides *highly experimental* support for running Twisted tests –
1024 tests that return a Deferred_ and rely on the Twisted reactor. You should not
1025 use this feature right now. We reserve the right to change the API and
1026 behaviour without telling you first.
1028 However, if you are going to, here's how you do it::
1030 from testtools import TestCase
1031 from testtools.deferredruntest import AsynchronousDeferredRunTest
1033 class MyTwistedTests(TestCase):
1035 run_tests_with = AsynchronousDeferredRunTest
1041 In particular, note that you do *not* have to use a special base ``TestCase``
1042 in order to run Twisted tests.
1044 You can also run individual tests within a test case class using the Twisted
1047 class MyTestsSomeOfWhichAreTwisted(TestCase):
1049 def test_normal(self):
1052 @run_test_with(AsynchronousDeferredRunTest)
1053 def test_twisted(self):
1057 Here are some tips for converting your Trial tests into testtools tests.
1059 * Use the ``AsynchronousDeferredRunTest`` runner
1060 * Make sure to upcall to ``setUp`` and ``tearDown``
1061 * Don't use ``setUpClass`` or ``tearDownClass``
1062 * Don't expect setting .todo, .timeout or .skip attributes to do anything
1063 * ``flushLoggedErrors`` is ``testtools.deferredruntest.flush_logged_errors``
1064 * ``assertFailure`` is ``testtools.deferredruntest.assert_fails_with``
1065 * Trial spins the reactor a couple of times before cleaning it up,
1066 ``AsynchronousDeferredRunTest`` does not. If you rely on this behavior, use
1067 ``AsynchronousDeferredRunTestForBrokenTwisted``.
1073 testtools comes with a few little things that make it a little bit easier to
1080 ``patch`` is a convenient way to monkey-patch a Python object for the duration
1081 of your test. It's especially useful for testing legacy code. e.g.::
1084 my_stream = StringIO()
1085 self.patch(sys, 'stderr', my_stream)
1086 run_some_code_that_prints_to_stderr()
1087 self.assertEqual('', my_stream.getvalue())
1089 The call to ``patch`` above masks ``sys.stderr`` with ``my_stream`` so that
1090 anything printed to stderr will be captured in a StringIO variable that can be
1091 actually tested. Once the test is done, the real ``sys.stderr`` is restored to
1098 Often when writing unit tests, you want to create an object that is a
1099 completely normal instance of its type. You don't want there to be anything
1100 special about its properties, because you are testing generic behaviour rather
1101 than specific conditions.
1103 A lot of the time, test authors do this by making up silly strings and numbers
1104 and passing them to constructors (e.g. 42, 'foo', "bar" etc), and that's
1105 fine. However, sometimes it's useful to be able to create arbitrary objects
1106 at will, without having to make up silly sample data.
1108 To help with this, ``testtools.TestCase`` implements creation methods called
1109 ``getUniqueString`` and ``getUniqueInteger``. They return strings and
1110 integers that are unique within the context of the test that can be used to
1111 assemble more complex objects. Here's a basic example where
1112 ``getUniqueString`` is used instead of saying "foo" or "bar" or whatever::
1114 class SomeTest(TestCase):
1116 def test_full_name(self):
1117 first_name = self.getUniqueString()
1118 last_name = self.getUniqueString()
1119 p = Person(first_name, last_name)
1120 self.assertEqual(p.full_name, "%s %s" % (first_name, last_name))
1123 And here's how it could be used to make a complicated test::
1125 class TestCoupleLogic(TestCase):
1127 def make_arbitrary_person(self):
1128 return Person(self.getUniqueString(), self.getUniqueString())
1130 def test_get_invitation(self):
1131 a = self.make_arbitrary_person()
1132 b = self.make_arbitrary_person()
1133 couple = Couple(a, b)
1134 event_name = self.getUniqueString()
1135 invitation = couple.get_invitation(event_name)
1138 "We invite %s and %s to %s" % (
1139 a.full_name, b.full_name, event_name))
1141 Essentially, creation methods like these are a way of reducing the number of
1142 assumptions in your tests and communicating to test readers that the exact
1143 details of certain variables don't actually matter.
1145 See pages 419-423 of `xUnit Test Patterns`_ by Gerard Meszaros for a detailed
1146 discussion of creation methods.
1155 Lots of the time we would like to conditionally import modules. testtools
1156 needs to do this itself, and graciously extends the ability to its users.
1161 from twisted.internet import defer
1167 defer = try_import('twisted.internet.defer')
1173 from StringIO import StringIO
1175 from io import StringIO
1179 StringIO = try_imports(['StringIO.StringIO', 'io.StringIO'])
1182 Safe attribute testing
1183 ----------------------
1185 ``hasattr`` is broken_ on many versions of Python. testtools provides
1186 ``safe_hasattr``, which can be used to safely test whether an object has a
1187 particular attribute.
1190 .. _testrepository: https://launchpad.net/testrepository
1191 .. _Trial: http://twistedmatrix.com/documents/current/core/howto/testing.html
1192 .. _nose: http://somethingaboutorange.com/mrl/projects/nose/
1193 .. _unittest2: http://pypi.python.org/pypi/unittest2
1194 .. _zope.testrunner: http://pypi.python.org/pypi/zope.testrunner/
1195 .. _xUnit test patterns: http://xunitpatterns.com/
1196 .. _fixtures: http://pypi.python.org/pypi/fixtures
1197 .. _unittest: http://docs.python.org/library/unittest.html
1198 .. _doctest: http://docs.python.org/library/doctest.html
1199 .. _Deferred: http://twistedmatrix.com/documents/current/core/howto/defer.html
1200 .. _discover: http://pypi.python.org/pypi/discover
1201 .. _`testtools API docs`: http://mumak.net/testtools/apidocs/
1202 .. _Distutils: http://docs.python.org/library/distutils.html
1203 .. _`setup configuration`: http://docs.python.org/distutils/configfile.html
1204 .. _broken: http://chipaca.com/post/3210673069/hasattr-17-less-harmful