4 Changes and improvements to testtools_, grouped by release.
16 * When an import error happens ``testtools.run`` will now show the full
17 error rather than just the name of the module that failed to import.
26 * ``testtools.TestCase`` now inherits from unittest2.TestCase, which
27 provides a ``setUpClass`` for upcalls on Python 2.6.
28 (Robert Collins, #1393283)
36 * Fixed our setup.py to use setup_requires to ensure the import dependencies
37 for testtools are present before setup.py runs (as setup.py imports testtools
38 to read out the version number). (Robert Collins)
40 * Support setUpClass skipping with self.skipException. Previously this worked
41 with unittest from 2.7 and above but was not supported by testtools - it was
42 a happy accident. Since we now hard depend on unittest2, we need to invert
43 our exception lookup priorities to support it. Regular skips done through
44 raise self.skipException will continue to work, since they were always caught
45 in our code - its because the suite type being used to implement setUpClass
46 has changed that an issue occured.
47 (Robert Collins, #1393068)
55 * Correctly express our unittest2 dependency: we don't work with old releases.
64 * Depends on unittest2 for discovery functionality and the ``TestProgram`` base
65 class. This brings in many fixes made to discovery where previously we were
66 only using the discovery package or the version in the release of Python
67 that the test execution was occuring on. (Robert Collins, #1271133)
69 * Fixed unit tests which were failing under pypy due to a change in the way
70 pypy formats tracebacks. (Thomi Richards)
72 * Fixed the testtools test suite to run correctly when run via ``unit2``
73 or ``testtools.run discover``.
75 * Make `testtools.content.text_content` error if anything other than text
76 is given as content. (Thomi Richards)
78 * We now publish wheels of testtools. (Robert Collins, #issue84)
86 * Exceptions in a ``fixture.getDetails`` method will no longer mask errors
87 raised from the same fixture's ``setUp`` method.
88 (Robert Collins, #1368440)
93 Long overdue, we've adopted a backwards compatibility statement and recognized
94 that we have plenty of users depending on our behaviour - calling our version
95 1.0.0 is a recognition of that.
100 * Fix a long-standing bug where tearDown and cleanUps would not be called if the
101 test run was interrupted. This should fix leaking external resources from
103 (Robert Collins, #1364188)
105 * Fix a long-standing bug where calling sys.exit(0) from within a test would
106 cause the test suite to exit with 0, without reporting a failure of that
107 test. We still allow the test suite to be exited (since catching higher order
108 exceptions requires exceptional circumstances) but we now call a last-resort
109 handler on the TestCase, resulting in an error being reported for the test.
110 (Robert Collins, #1364188)
112 * Fix an issue where tests skipped with the ``skip``* family of decorators would
113 still have their ``setUp`` and ``tearDown`` functions called.
114 (Thomi Richards, #https://github.com/testing-cabal/testtools/issues/86)
116 * We have adopted a formal backwards compatibility statement (see hacking.rst)
122 Brown paper bag release - 0.9.38 was broken for some users,
123 _jython_aware_splitext was not defined entirely compatibly.
124 (Robert Collins, #https://github.com/testing-cabal/testtools/issues/100)
129 Bug fixes for test importing.
134 * Discovery import error detection wasn't implemented for python 2.6 (the
135 'discover' module). (Robert Collins)
137 * Discovery now executes load_tests (if present) in __init__ in all packages.
138 (Robert Collins, http://bugs.python.org/issue16662)
143 Minor improvements to correctness.
148 * ``stdout`` is now correctly honoured on ``run.TestProgram`` - before the
149 runner objects would be created with no stdout parameter. If construction
150 fails, the previous parameter list is attempted, permitting compatibility
151 with Runner classes that don't accept stdout as a parameter.
154 * The ``ExtendedToStreamDecorator`` now handles content objects with one less
155 packet - the last packet of the source content is sent with EOF set rather
156 than an empty packet with EOF set being sent after the last packet of the
157 source content. (Robert Collins)
162 Welcome to our long overdue 0.9.36 release, which improves compatibility with
163 Python3.4, adds assert_that, a function for using matchers without TestCase
164 objects, and finally will error if you try to use setUp or tearDown twice -
165 since that invariably leads to bad things of one sort or another happening.
170 * Error if ``setUp`` or ``tearDown`` are called twice.
171 (Robert Collins, #882884)
173 * Make testtools compatible with the ``unittest.expectedFailure`` decorator in
174 Python 3.4. (Thomi Richards)
180 * Introduce the assert_that function, which allows matchers to be used
181 independent of testtools.TestCase. (Daniel Watkins, #1243834)
190 * Removed a number of code paths where Python 2.4 and Python 2.5 were
191 explicitly handled. (Daniel Watkins)
196 * Added the ``testtools.TestCase.expectThat`` method, which implements
197 delayed assertions. (Thomi Richards)
199 * Docs are now built as part of the Travis-CI build, reducing the chance of
200 Read The Docs being broken accidentally. (Daniel Watkins, #1158773)
208 * Added ability for ``testtools.TestCase`` instances to force a test to
209 fail, even if no assertions failed. (Thomi Richards)
211 * Added ``testtools.content.StacktraceContent``, a content object that
212 automatically creates a ``StackLinesContent`` object containing the current
213 stack trace. (Thomi Richards)
215 * ``AnyMatch`` is now exported properly in ``testtools.matchers``.
216 (Robert Collins, Rob Kennedy, github #44)
218 * In Python 3.3, if there are duplicate test ids, tests.sort() will
219 fail and raise TypeError. Detect the duplicate test ids firstly in
220 sorted_tests() to ensure that all test ids are unique.
223 * ``json_content`` is now in the ``__all__`` attribute for
224 ``testtools.content``. (Robert Collins)
226 * Network tests now bind to 127.0.0.1 to avoid (even temporary) network
227 visible ports. (Benedikt Morbach, github #46)
229 * Test listing now explicitly indicates by printing 'Failed to import' and
230 exiting (2) when an import has failed rather than only signalling through the
231 test name. (Robert Collins, #1245672)
233 * ``test_compat.TestDetectEncoding.test_bom`` now works on Python 3.3 - the
234 corner case with euc_jp is no longer permitted in Python 3.3 so we can
235 skip it. (Martin [gz], #1251962)
243 * Added ``addDetailuniqueName`` method to ``testtools.TestCase`` class.
246 * Removed some unused code from ``testtools.content.TracebackContent``.
249 * Added ``testtools.StackLinesContent``: a content object for displaying
250 pre-processed stack lines. (Thomi Richards)
252 * ``StreamSummary`` was calculating testsRun incorrectly: ``exists`` status
253 tests were counted as run tests, but they are not.
254 (Robert Collins, #1203728)
259 Regular maintenance release. Special thanks to new contributor, Xiao Hanyu!
264 * ``testttols.compat._format_exc_info`` has been refactored into several
265 smaller functions. (Thomi Richards)
270 * Stacktrace filtering no longer hides unittest frames that are surrounded by
271 user frames. We will reenable this when we figure out a better algorithm for
272 retaining meaning. (Robert Collins, #1188420)
274 * The compatibility code for skipped tests with unittest2 was broken.
275 (Robert Collins, #1190951)
277 * Various documentation improvements (Clint Byrum, Xiao Hanyu).
285 * ``ExpectedException`` now accepts a msg parameter for describing an error,
286 much the same as assertEquals etc. (Robert Collins)
291 A new sort of TestResult, the StreamResult has been added, as a prototype for
292 a revised standard library test result API. Expect this API to change.
293 Although we will try to preserve compatibility for early adopters, it is
294 experimental and we might need to break it if it turns out to be unsuitable.
298 * ``assertRaises`` works properly for exception classes that have custom
301 * ``ConcurrentTestSuite`` was silently eating exceptions that propagate from
302 the test.run(result) method call. Ignoring them is fine in a normal test
303 runner, but when they happen in a different thread, the thread that called
304 suite.run() is not in the stack anymore, and the exceptions are lost. We now
305 create a synthetic test recording any such exception.
306 (Robert Collins, #1130429)
308 * Fixed SyntaxError raised in ``_compat2x.py`` when installing via Python 3.
311 * New class ``StreamResult`` which defines the API for the new result type.
314 * New support class ``ConcurrentStreamTestSuite`` for convenient construction
315 and utilisation of ``StreamToQueue`` objects. (Robert Collins)
317 * New support class ``CopyStreamResult`` which forwards events onto multiple
318 ``StreamResult`` objects (each of which receives all the events).
321 * New support class ``StreamSummary`` which summarises a ``StreamResult``
322 stream compatibly with ``TestResult`` code. (Robert Collins)
324 * New support class ``StreamTagger`` which adds or removes tags from
325 ``StreamResult`` events. (RobertCollins)
327 * New support class ``StreamToDict`` which converts a ``StreamResult`` to a
328 series of dicts describing a test. Useful for writing trivial stream
329 analysers. (Robert Collins)
331 * New support class ``TestControl`` which permits cancelling an in-progress
332 run. (Robert Collins)
334 * New support class ``StreamFailFast`` which calls a ``TestControl`` instance
335 to abort the test run when a failure is detected. (Robert Collins)
337 * New support class ``ExtendedToStreamDecorator`` which translates both regular
338 unittest TestResult API calls and the ExtendedTestResult API which testtools
339 has supported into the StreamResult API. ExtendedToStreamDecorator also
340 forwards calls made in the StreamResult API, permitting it to be used
341 anywhere a StreamResult is used. Key TestResult query methods like
342 wasSuccessful and shouldStop are synchronised with the StreamResult API
343 calls, but the detailed statistics like the list of errors are not - a
344 separate consumer will be created to support that.
347 * New support class ``StreamToExtendedDecorator`` which translates
348 ``StreamResult`` API calls into ``ExtendedTestResult`` (or any older
349 ``TestResult``) calls. This permits using un-migrated result objects with
350 new runners / tests. (Robert Collins)
352 * New support class ``StreamToQueue`` for sending messages to one
353 ``StreamResult`` from multiple threads. (Robert Collins)
355 * New support class ``TimestampingStreamResult`` which adds a timestamp to
356 events with no timestamp. (Robert Collins)
358 * New ``TestCase`` decorator ``DecorateTestCaseResult`` that adapts the
359 ``TestResult`` or ``StreamResult`` a case will be run with, for ensuring that
360 a particular result object is used even if the runner running the test doesn't
361 know to use it. (Robert Collins)
363 * New test support class ``testtools.testresult.doubles.StreamResult``, which
364 captures all the StreamResult events. (Robert Collins)
366 * ``PlaceHolder`` can now hold tags, and applies them before, and removes them
367 after, the test. (Robert Collins)
369 * ``PlaceHolder`` can now hold timestamps, and applies them before the test and
370 then before the outcome. (Robert Collins)
372 * ``StreamResultRouter`` added. This is useful for demultiplexing - e.g. for
373 partitioning analysis of events or sending feedback encapsulated in
374 StreamResult events back to their source. (Robert Collins)
376 * ``testtools.run.TestProgram`` now supports the ``TestRunner`` taking over
377 responsibility for formatting the output of ``--list-tests``.
380 * The error message for setUp and tearDown upcall errors was broken on Python
381 3.4. (Monty Taylor, Robert Collins, #1140688)
383 * The repr of object() on pypy includes the object id, which was breaking a
384 test that accidentally depended on the CPython repr for object().
390 A simple bug fix, and better error messages when you don't up-call.
395 * ``testtools.content_type.ContentType`` incorrectly used ',' rather than ';'
396 to separate parameters. (Robert Collins)
401 * ``testtools.compat.unicode_output_stream`` was wrapping a stream encoder
402 around ``io.StringIO`` and ``io.TextIOWrapper`` objects, which was incorrect.
405 * Report the name of the source file for setUp and tearDown upcall errors.
411 Testtools has moved VCS - https://github.com/testing-cabal/testtools/ is
412 the new home. Bug tracking is still on Launchpad, and releases are on Pypi.
414 We made this change to take advantage of the richer ecosystem of tools around
415 Git, and to lower the barrier for new contributors.
420 * New ``testtools.testcase.attr`` and ``testtools.testcase.WithAttributes``
421 helpers allow marking up test case methods with simple labels. This permits
422 filtering tests with more granularity than organising them into modules and
423 test classes. (Robert Collins)
431 * New matcher ``HasLength`` for matching the length of a collection.
434 * New matcher ``MatchesPredicateWithParams`` make it still easier to create
435 ad hoc matchers. (Robert Collins)
437 * We have a simpler release process in future - see doc/hacking.rst.
443 Brown paper bag fix: failed to document the need for setup to be able to use
444 extras. Compounded by pip not supporting setup_requires.
449 * setup.py now can generate egg_info even if extras is not available.
450 Also lists extras in setup_requires for easy_install.
451 (Robert Collins, #1102464)
459 * ``python -m testtools.run --load-list`` will now preserve any custom suites
460 (such as ``testtools.FixtureSuite`` or ``testresources.OptimisingTestSuite``)
461 rather than flattening them.
462 (Robert Collins, #827175)
464 * Testtools now depends on extras, a small library split out from it to contain
465 generally useful non-testing facilities. Since extras has been around for a
466 couple of testtools releases now, we're making this into a hard dependency of
467 testtools. (Robert Collins)
469 * Testtools now uses setuptools rather than distutils so that we can document
470 the extras dependency. (Robert Collins)
475 * Testtools will no longer override test code registered details called
476 'traceback' when reporting caught exceptions from test code.
477 (Robert Collins, #812793)
485 * ``testtools.run discover`` will now sort the tests it discovered. This is a
486 workaround for http://bugs.python.org/issue16709. Non-standard test suites
487 are preserved, and their ``sort_tests()`` method called (if they have such an
488 attribute). ``testtools.testsuite.sorted_tests(suite, True)`` can be used by
489 such suites to do a local sort. (Robert Collins, #1091512)
491 * ``ThreadsafeForwardingResult`` now defines a stub ``progress`` method, which
492 fixes ``testr run`` of streams containing progress markers (by discarding the
493 progress data). (Robert Collins, #1019165)
501 * ``run.TestToolsTestRunner`` now accepts the verbosity, buffer and failfast
502 arguments the upstream python TestProgram code wants to give it, making it
503 possible to support them in a compatible fashion. (Robert Collins)
508 * ``testtools.run`` now supports the ``-f`` or ``--failfast`` parameter.
509 Previously it was advertised in the help but ignored.
510 (Robert Collins, #1090582)
512 * ``AnyMatch`` added, a new matcher that matches when any item in a collection
513 matches the given matcher. (Jonathan Lange)
515 * Spelling corrections to documentation. (Vincent Ladeuil)
517 * ``TestProgram`` now has a sane default for its ``testRunner`` argument.
520 * The test suite passes on Python 3 again. (Robert Collins)
528 * ``content_from_file`` and ``content_from_stream`` now accept seek_offset and
529 seek_whence parameters allowing them to be used to grab less than the full
530 stream, or to be used with StringIO streams. (Robert Collins, #1088693)
538 * ``DirContains`` correctly exposed, after being accidentally hidden in the
539 great matcher re-organization of 0.9.17. (Jonathan Lange)
545 Three new matchers that'll rock your world.
550 * New, powerful matchers that match items in a dictionary:
552 - ``MatchesDict``, match every key in a dictionary with a key in a
553 dictionary of matchers. For when the set of expected keys is equal to the
554 set of observed keys.
556 - ``ContainsDict``, every key in a dictionary of matchers must be
557 found in a dictionary, and the values for those keys must match. For when
558 the set of expected keys is a subset of the set of observed keys.
560 - ``ContainedByDict``, every key in a dictionary must be found in
561 a dictionary of matchers. For when the set of expected keys is a superset
562 of the set of observed keys.
564 The names are a little confusing, sorry. We're still trying to figure out
565 how to present the concept in the simplest way possible.
571 How embarrassing! Three releases in two days.
573 We've worked out the kinks and have confirmation from our downstreams that
574 this is all good. Should be the last release for a little while. Please
575 ignore 0.9.18 and 0.9.17.
580 * Include the matcher tests in the release, allowing the tests to run and
581 pass from the release tarball. (Jonathan Lange)
583 * Fix cosmetic test failures in Python 3.3, introduced during release 0.9.17.
590 Due to an oversight, release 0.9.18 did not contain the new
591 ``testtools.matchers`` package and was thus completely broken. This release
592 corrects that, returning us all to normality.
597 This release brings better discover support and Python3.x improvements. There
598 are still some test failures on Python3.3 but they are cosmetic - the library
599 is as usable there as on any other Python 3 release.
604 * The ``testtools.matchers`` package has been split up. No change to the
605 public interface. (Jonathan Lange)
610 * ``python -m testtools.run discover . --list`` now works. (Robert Collins)
612 * Correctly handling of bytes vs text in JSON content type. (Martin [gz])
618 Some new matchers and a new content helper for JSON content.
620 This is the first release of testtools to drop support for Python 2.4 and 2.5.
621 If you need support for either of those versions, please use testtools 0.9.15.
626 * New content helper, ``json_content`` (Jonathan Lange)
630 * ``ContainsAll`` for asserting one thing is a subset of another
633 * ``SameMembers`` for asserting two iterators have the same members.
636 * Reraising of exceptions in Python 3 is more reliable. (Martin [gz])
642 This is the last release to support Python2.4 and 2.5. It brings in a slew of
643 improvements to test tagging and concurrency, making running large test suites
644 with partitioned workers more reliable and easier to reproduce exact test
645 ordering in a given worker. See our sister project ``testrepository`` for a
646 test runner that uses these features.
651 * ``PlaceHolder`` and ``ErrorHolder`` now support being given result details.
654 * ``ErrorHolder`` is now just a function - all the logic is in ``PlaceHolder``.
657 * ``TestResult`` and all other ``TestResult``-like objects in testtools
658 distinguish between global tags and test-local tags, as per the subunit
659 specification. (Jonathan Lange)
661 * This is the **last** release of testtools that supports Python 2.4 or 2.5.
662 These releases are no longer supported by the Python community and do not
663 receive security updates. If this affects you, you will need to either
664 stay on this release or perform your own backports.
665 (Jonathan Lange, Robert Collins)
667 * ``ThreadsafeForwardingResult`` now forwards global tags as test-local tags,
668 making reasoning about the correctness of the multiplexed stream simpler.
669 This preserves the semantic value (what tags apply to a given test) while
670 consuming less stream size (as no negative-tag statement is needed).
671 (Robert Collins, Gary Poster, #986434)
676 * API documentation corrections. (Raphaël Badin)
678 * ``ConcurrentTestSuite`` now takes an optional ``wrap_result`` parameter
679 that can be used to wrap the ``ThreadsafeForwardingResults`` created by
680 the suite. (Jonathan Lange)
682 * ``Tagger`` added. It's a new ``TestResult`` that tags all tests sent to
683 it with a particular set of tags. (Jonathan Lange)
685 * ``testresultdecorator`` brought over from subunit. (Jonathan Lange)
687 * All ``TestResult`` wrappers now correctly forward ``current_tags`` from
688 their wrapped results, meaning that ``current_tags`` can always be relied
689 upon to return the currently active tags on a test result.
691 * ``TestByTestResult``, a ``TestResult`` that calls a method once per test,
692 added. (Jonathan Lange)
694 * ``ThreadsafeForwardingResult`` correctly forwards ``tags()`` calls where
695 only one of ``new_tags`` or ``gone_tags`` are specified.
696 (Jonathan Lange, #980263)
698 * ``ThreadsafeForwardingResult`` no longer leaks local tags from one test
699 into all future tests run. (Jonathan Lange, #985613)
701 * ``ThreadsafeForwardingResult`` has many, many more tests. (Jonathan Lange)
707 Our sister project, `subunit <https://launchpad.net/subunit>`_, was using a
708 private API that was deleted in the 0.9.13 release. This release restores
709 that API in order to smooth out the upgrade path.
711 If you don't use subunit, then this release won't matter very much to you.
717 Plenty of new matchers and quite a few critical bug fixes (especially to do
718 with stack traces from failed assertions). A net win for all.
723 * ``MatchesAll`` now takes an ``first_only`` keyword argument that changes how
724 mismatches are displayed. If you were previously passing matchers to
725 ``MatchesAll`` with keyword arguments, then this change might affect your
726 test results. (Jonathan Lange)
731 * Actually hide all of the testtools stack for assertion failures. The
732 previous release promised clean stack, but now we actually provide it.
733 (Jonathan Lange, #854769)
735 * ``assertRaises`` now includes the ``repr`` of the callable that failed to raise
736 properly. (Jonathan Lange, #881052)
738 * Asynchronous tests no longer hang when run with trial.
739 (Jonathan Lange, #926189)
741 * ``Content`` objects now have an ``as_text`` method to convert their contents
742 to Unicode text. (Jonathan Lange)
744 * Failed equality assertions now line up. (Jonathan Lange, #879339)
746 * ``FullStackRunTest`` no longer aborts the test run if a test raises an
747 error. (Jonathan Lange)
749 * ``MatchesAll`` and ``MatchesListwise`` both take a ``first_only`` keyword
750 argument. If True, they will report only on the first mismatch they find,
751 and not continue looking for other possible mismatches.
754 * New helper, ``Nullary`` that turns callables with arguments into ones that
755 don't take arguments. (Jonathan Lange)
759 * ``DirContains`` matches the contents of a directory.
760 (Jonathan Lange, James Westby)
762 * ``DirExists`` matches if a directory exists.
763 (Jonathan Lange, James Westby)
765 * ``FileContains`` matches the contents of a file.
766 (Jonathan Lange, James Westby)
768 * ``FileExists`` matches if a file exists.
769 (Jonathan Lange, James Westby)
771 * ``HasPermissions`` matches the permissions of a file. (Jonathan Lange)
773 * ``MatchesPredicate`` matches if a predicate is true. (Jonathan Lange)
775 * ``PathExists`` matches if a path exists. (Jonathan Lange, James Westby)
777 * ``SamePath`` matches if two paths are the same. (Jonathan Lange)
779 * ``TarballContains`` matches the contents of a tarball. (Jonathan Lange)
781 * ``MultiTestResult`` supports the ``tags`` method.
782 (Graham Binns, Francesco Banconi, #914279)
784 * ``ThreadsafeForwardingResult`` supports the ``tags`` method.
785 (Graham Binns, Francesco Banconi, #914279)
787 * ``ThreadsafeForwardingResult`` no longer includes semaphore acquisition time
788 in the test duration (for implicitly timed test runs).
789 (Robert Collins, #914362)
794 This is a very big release. We've made huge improvements on three fronts:
795 1. Test failures are way nicer and easier to read
796 2. Matchers and ``assertThat`` are much more convenient to use
797 3. Correct handling of extended unicode characters
799 We've trimmed off the fat from the stack trace you get when tests fail, we've
800 cut out the bits of error messages that just didn't help, we've made it easier
801 to annotate mismatch failures, to compare complex objects and to match raised
804 Testing code was never this fun.
809 * ``AfterPreproccessing`` renamed to ``AfterPreprocessing``, which is a more
810 correct spelling. Old name preserved for backwards compatibility, but is
811 now deprecated. Please stop using it.
812 (Jonathan Lange, #813460)
814 * ``assertThat`` raises ``MismatchError`` instead of
815 ``TestCase.failureException``. ``MismatchError`` is a subclass of
816 ``AssertionError``, so in most cases this change will not matter. However,
817 if ``self.failureException`` has been set to a non-default value, then
818 mismatches will become test errors rather than test failures.
820 * ``gather_details`` takes two dicts, rather than two detailed objects.
821 (Jonathan Lange, #801027)
823 * ``MatchesRegex`` mismatch now says "<value> does not match /<regex>/" rather
824 than "<regex> did not match <value>". The regular expression contains fewer
825 backslashes too. (Jonathan Lange, #818079)
827 * Tests that run with ``AsynchronousDeferredRunTest`` now have the ``reactor``
828 attribute set to the running reactor. (Jonathan Lange, #720749)
833 * All public matchers are now in ``testtools.matchers.__all__``.
834 (Jonathan Lange, #784859)
836 * ``assertThat`` can actually display mismatches and matchers that contain
837 extended unicode characters. (Jonathan Lange, Martin [gz], #804127)
839 * ``assertThat`` output is much less verbose, displaying only what the mismatch
840 tells us to display. Old-style verbose output can be had by passing
841 ``verbose=True`` to assertThat. (Jonathan Lange, #675323, #593190)
843 * ``assertThat`` accepts a message which will be used to annotate the matcher.
844 This can be given as a third parameter or as a keyword parameter.
847 * Automated the Launchpad part of the release process.
848 (Jonathan Lange, #623486)
850 * Correctly display non-ASCII unicode output on terminals that claim to have a
851 unicode encoding. (Martin [gz], #804122)
853 * ``DocTestMatches`` correctly handles unicode output from examples, rather
854 than raising an error. (Martin [gz], #764170)
856 * ``ErrorHolder`` and ``PlaceHolder`` added to docs. (Jonathan Lange, #816597)
858 * ``ExpectedException`` now matches any exception of the given type by
859 default, and also allows specifying a ``Matcher`` rather than a mere regular
860 expression. (Jonathan Lange, #791889)
862 * ``FixtureSuite`` added, allows test suites to run with a given fixture.
865 * Hide testtools's own stack frames when displaying tracebacks, making it
866 easier for test authors to focus on their errors.
867 (Jonathan Lange, Martin [gz], #788974)
869 * Less boilerplate displayed in test failures and errors.
870 (Jonathan Lange, #660852)
872 * ``MatchesException`` now allows you to match exceptions against any matcher,
873 rather than just regular expressions. (Jonathan Lange, #791889)
875 * ``MatchesException`` now permits a tuple of types rather than a single type
876 (when using the type matching mode). (Robert Collins)
878 * ``MatchesStructure.byEquality`` added to make the common case of matching
879 many attributes by equality much easier. ``MatchesStructure.byMatcher``
880 added in case folk want to match by things other than equality.
883 * New convenience assertions, ``assertIsNone`` and ``assertIsNotNone``.
888 * ``AllMatch`` matches many values against a single matcher.
889 (Jonathan Lange, #615108)
891 * ``Contains``. (Robert Collins)
893 * ``GreaterThan``. (Christian Kampka)
895 * New helper, ``safe_hasattr`` added. (Jonathan Lange)
897 * ``reraise`` added to ``testtools.compat``. (Jonathan Lange)
903 This release brings consistent use of super for better compatibility with
904 multiple inheritance, fixed Python3 support, improvements in fixture and mather
905 outputs and a compat helper for testing libraries that deal with bytestrings.
910 * ``TestCase`` now uses super to call base ``unittest.TestCase`` constructor,
911 ``setUp`` and ``tearDown``. (Tim Cole, #771508)
913 * If, when calling ``useFixture`` an error occurs during fixture set up, we
914 still attempt to gather details from the fixture. (Gavin Panella)
920 * Additional compat helper for ``BytesIO`` for libraries that build on
921 testtools and are working on Python 3 porting. (Robert Collins)
923 * Corrected documentation for ``MatchesStructure`` in the test authors
924 document. (Jonathan Lange)
926 * ``LessThan`` error message now says something that is logically correct.
927 (Gavin Panella, #762008)
929 * Multiple details from a single fixture are now kept separate, rather than
930 being mooshed together. (Gavin Panella, #788182)
932 * Python 3 support now back in action. (Martin [gz], #688729)
934 * ``try_import`` and ``try_imports`` have a callback that is called whenever
935 they fail to import a module. (Martin Pool)
941 The last release of testtools could not be easy_installed. This is considered
942 severe enough for a re-release.
947 * Include ``doc/`` in the source distribution, making testtools installable
948 from PyPI again (Tres Seaver, #757439)
954 Many, many new matchers, vastly expanded documentation, stacks of bug fixes,
955 better unittest2 integration. If you've ever wanted to try out testtools but
956 been afraid to do so, this is the release to try.
962 * The timestamps generated by ``TestResult`` objects when no timing data has
963 been received are now datetime-with-timezone, which allows them to be
964 sensibly serialised and transported. (Robert Collins, #692297)
969 * ``AnnotatedMismatch`` now correctly returns details.
970 (Jonathan Lange, #724691)
972 * distutils integration for the testtools test runner. Can now use it for
973 'python setup.py test'. (Christian Kampka, #693773)
975 * ``EndsWith`` and ``KeysEqual`` now in testtools.matchers.__all__.
976 (Jonathan Lange, #692158)
978 * ``MatchesException`` extended to support a regular expression check against
979 the str() of a raised exception. (Jonathan Lange)
981 * ``MultiTestResult`` now forwards the ``time`` API. (Robert Collins, #692294)
983 * ``MultiTestResult`` now documented in the manual. (Jonathan Lange, #661116)
985 * New content helpers ``content_from_file``, ``content_from_stream`` and
986 ``attach_file`` make it easier to attach file-like objects to a
987 test. (Jonathan Lange, Robert Collins, #694126)
989 * New ``ExpectedException`` context manager to help write tests against things
990 that are expected to raise exceptions. (Aaron Bentley)
994 * ``MatchesListwise`` matches an iterable of matchers against an iterable
995 of values. (Michael Hudson-Doyle)
997 * ``MatchesRegex`` matches a string against a regular expression.
998 (Michael Hudson-Doyle)
1000 * ``MatchesStructure`` matches attributes of an object against given
1001 matchers. (Michael Hudson-Doyle)
1003 * ``AfterPreproccessing`` matches values against a matcher after passing them
1004 through a callable. (Michael Hudson-Doyle)
1006 * ``MatchesSetwise`` matches an iterable of matchers against an iterable of
1007 values, without regard to order. (Michael Hudson-Doyle)
1009 * ``setup.py`` can now build a snapshot when Bazaar is installed but the tree
1010 is not a Bazaar tree. (Jelmer Vernooij)
1012 * Support for running tests using distutils (Christian Kampka, #726539)
1014 * Vastly improved and extended documentation. (Jonathan Lange)
1016 * Use unittest2 exception classes if available. (Jelmer Vernooij)
1022 In this release we bring some very interesting improvements:
1024 * new matchers for exceptions, sets, lists, dicts and more.
1026 * experimental (works but the contract isn't supported) twisted reactor
1029 * The built in runner can now list tests and filter tests (the -l and
1030 --load-list options).
1035 * addUnexpectedSuccess is translated to addFailure for test results that don't
1036 know about addUnexpectedSuccess. Further, it fails the entire result for
1037 all testtools TestResults (i.e. wasSuccessful() returns False after
1038 addUnexpectedSuccess has been called). Note that when using a delegating
1039 result such as ThreadsafeForwardingResult, MultiTestResult or
1040 ExtendedToOriginalDecorator then the behaviour of addUnexpectedSuccess is
1041 determined by the delegated to result(s).
1042 (Jonathan Lange, Robert Collins, #654474, #683332)
1044 * startTestRun will reset any errors on the result. That is, wasSuccessful()
1045 will always return True immediately after startTestRun() is called. This
1046 only applies to delegated test results (ThreadsafeForwardingResult,
1047 MultiTestResult and ExtendedToOriginalDecorator) if the delegated to result
1048 is a testtools test result - we cannot reliably reset the state of unknown
1049 test result class instances. (Jonathan Lange, Robert Collins, #683332)
1051 * Responsibility for running test cleanups has been moved to ``RunTest``.
1052 This change does not affect public APIs and can be safely ignored by test
1053 authors. (Jonathan Lange, #662647)
1060 * ``EndsWith`` which complements the existing ``StartsWith`` matcher.
1061 (Jonathan Lange, #669165)
1063 * ``MatchesException`` matches an exception class and parameters. (Robert
1066 * ``KeysEqual`` matches a dictionary with particular keys. (Jonathan Lange)
1068 * ``assertIsInstance`` supports a custom error message to be supplied, which
1069 is necessary when using ``assertDictEqual`` on Python 2.7 with a
1070 ``testtools.TestCase`` base class. (Jelmer Vernooij)
1072 * Experimental support for running tests that return Deferreds.
1073 (Jonathan Lange, Martin [gz])
1075 * Provide a per-test decorator, run_test_with, to specify which RunTest
1076 object to use for a given test. (Jonathan Lange, #657780)
1078 * Fix the runTest parameter of TestCase to actually work, rather than raising
1079 a TypeError. (Jonathan Lange, #657760)
1081 * Non-release snapshots of testtools will now work with buildout.
1082 (Jonathan Lange, #613734)
1084 * Malformed SyntaxErrors no longer blow up the test suite. (Martin [gz])
1086 * ``MismatchesAll.describe`` no longer appends a trailing newline.
1087 (Michael Hudson-Doyle, #686790)
1089 * New helpers for conditionally importing modules, ``try_import`` and
1090 ``try_imports``. (Jonathan Lange)
1092 * ``Raises`` added to the ``testtools.matchers`` module - matches if the
1093 supplied callable raises, and delegates to an optional matcher for validation
1094 of the exception. (Robert Collins)
1096 * ``raises`` added to the ``testtools.matchers`` module - matches if the
1097 supplied callable raises and delegates to ``MatchesException`` to validate
1098 the exception. (Jonathan Lange)
1100 * Tests will now pass on Python 2.6.4 : an ``Exception`` change made only in
1101 2.6.4 and reverted in Python 2.6.5 was causing test failures on that version.
1102 (Martin [gz], #689858).
1104 * ``testtools.TestCase.useFixture`` has been added to glue with fixtures nicely.
1107 * ``testtools.run`` now supports ``-l`` to list tests rather than executing
1108 them. This is useful for integration with external test analysis/processing
1109 tools like subunit and testrepository. (Robert Collins)
1111 * ``testtools.run`` now supports ``--load-list``, which takes a file containing
1112 test ids, one per line, and intersects those ids with the tests found. This
1113 allows fine grained control of what tests are run even when the tests cannot
1114 be named as objects to import (e.g. due to test parameterisation via
1115 testscenarios). (Robert Collins)
1117 * Update documentation to say how to use testtools.run() on Python 2.4.
1118 (Jonathan Lange, #501174)
1120 * ``text_content`` conveniently converts a Python string to a Content object.
1121 (Jonathan Lange, James Westby)
1128 Lots of little cleanups in this release; many small improvements to make your
1129 testing life more pleasant.
1134 * Cleanups can raise ``testtools.MultipleExceptions`` if they have multiple
1135 exceptions to report. For instance, a cleanup which is itself responsible for
1136 running several different internal cleanup routines might use this.
1138 * Code duplication between assertEqual and the matcher Equals has been removed.
1140 * In normal circumstances, a TestCase will no longer share details with clones
1141 of itself. (Andrew Bennetts, bug #637725)
1143 * Less exception object cycles are generated (reduces peak memory use between
1144 garbage collection). (Martin [gz])
1146 * New matchers 'DoesNotStartWith' and 'StartsWith' contributed by Canonical
1147 from the Launchpad project. Written by James Westby.
1149 * Timestamps as produced by subunit protocol clients are now forwarded in the
1150 ThreadsafeForwardingResult so correct test durations can be reported.
1151 (Martin [gz], Robert Collins, #625594)
1153 * With unittest from Python 2.7 skipped tests will now show only the reason
1154 rather than a serialisation of all details. (Martin [gz], #625583)
1156 * The testtools release process is now a little better documented and a little
1157 smoother. (Jonathan Lange, #623483, #623487)
1163 Nothing major in this release, just enough small bits and pieces to make it
1164 useful enough to upgrade to.
1166 In particular, a serious bug in assertThat() has been fixed, it's easier to
1167 write Matchers, there's a TestCase.patch() method for those inevitable monkey
1168 patches and TestCase.assertEqual gives slightly nicer errors.
1173 * 'TestCase.assertEqual' now formats errors a little more nicely, in the
1176 * Added `PlaceHolder` and `ErrorHolder`, TestCase-like objects that can be
1177 used to add results to a `TestResult`.
1179 * 'Mismatch' now takes optional description and details parameters, so
1180 custom Matchers aren't compelled to make their own subclass.
1182 * jml added a built-in UTF8_TEXT ContentType to make it slightly easier to
1183 add details to test results. See bug #520044.
1185 * Fix a bug in our built-in matchers where assertThat would blow up if any
1186 of them failed. All built-in mismatch objects now provide get_details().
1188 * New 'Is' matcher, which lets you assert that a thing is identical to
1191 * New 'LessThan' matcher which lets you assert that a thing is less than
1194 * TestCase now has a 'patch()' method to make it easier to monkey-patching
1195 objects in tests. See the manual for more information. Fixes bug #310770.
1197 * MultiTestResult methods now pass back return values from the results it
1203 This release fixes some obscure traceback formatting issues that probably
1204 weren't affecting you but were certainly breaking our own test suite.
1209 * Jamu Kakar has updated classes in testtools.matchers and testtools.runtest
1210 to be new-style classes, fixing bug #611273.
1215 * Martin[gz] fixed traceback handling to handle cases where extract_tb returns
1216 a source line of None. Fixes bug #611307.
1218 * Martin[gz] fixed an unicode issue that was causing the tests to fail,
1219 closing bug #604187.
1221 * testtools now handles string exceptions (although why would you want to use
1222 them?) and formats their tracebacks correctly. Thanks to Martin[gz] for
1228 This release overhauls the traceback formatting layer to deal with Python 2
1229 line numbers and traceback objects often being local user encoded strings
1230 rather than unicode objects. Test discovery has also been added and Python 3.1
1231 is also supported. Finally, the Mismatch protocol has been extended to let
1232 Matchers collaborate with tests in supplying detailed data about failures.
1237 * testtools.utils has been renamed to testtools.compat. Importing
1238 testtools.utils will now generate a deprecation warning.
1243 * Add machinery for Python 2 to create unicode tracebacks like those used by
1244 Python 3. This means testtools no longer throws on encountering non-ascii
1245 filenames, source lines, or exception strings when displaying test results.
1246 Largely contributed by Martin[gz] with some tweaks from Robert Collins.
1248 * James Westby has supplied test discovery support using the Python 2.7
1249 TestRunner in testtools.run. This requires the 'discover' module. This
1252 * Python 3.1 is now supported, thanks to Martin[gz] for a partial patch.
1253 This fixes bug #592375.
1255 * TestCase.addCleanup has had its docstring corrected about when cleanups run.
1257 * TestCase.skip is now deprecated in favour of TestCase.skipTest, which is the
1258 Python2.7 spelling for skip. This closes bug #560436.
1260 * Tests work on IronPython patch from Martin[gz] applied.
1262 * Thanks to a patch from James Westby testtools.matchers.Mismatch can now
1263 supply a get_details method, which assertThat will query to provide
1264 additional attachments. This can be used to provide additional detail
1265 about the mismatch that doesn't suite being included in describe(). For
1266 instance, if the match process was complex, a log of the process could be
1267 included, permitting debugging.
1269 * testtools.testresults.real._StringException will now answer __str__ if its
1270 value is unicode by encoding with UTF8, and vice versa to answer __unicode__.
1271 This permits subunit decoded exceptions to contain unicode and still format
1277 More matchers, Python 2.4 support, faster test cloning by switching to copy
1278 rather than deepcopy and better output when exceptions occur in cleanups are
1279 the defining characteristics of this release.
1284 * New matcher "Annotate" that adds a simple string message to another matcher,
1285 much like the option 'message' parameter to standard library assertFoo
1288 * New matchers "Not" and "MatchesAll". "Not" will invert another matcher, and
1289 "MatchesAll" that needs a successful match for all of its arguments.
1291 * On Python 2.4, where types.FunctionType cannot be deepcopied, testtools will
1292 now monkeypatch copy._deepcopy_dispatch using the same trivial patch that
1293 added such support to Python 2.5. The monkey patch is triggered by the
1294 absence of FunctionType from the dispatch dict rather than a version check.
1297 * On windows the test 'test_now_datetime_now' should now work reliably.
1299 * TestCase.getUniqueInteger and TestCase.getUniqueString now have docstrings.
1301 * TestCase.getUniqueString now takes an optional prefix parameter, so you can
1302 now use it in circumstances that forbid strings with '.'s, and such like.
1304 * testtools.testcase.clone_test_with_new_id now uses copy.copy, rather than
1305 copy.deepcopy. Tests that need a deeper copy should use the copy protocol to
1306 control how they are copied. Bug #498869.
1308 * The backtrace test result output tests should now pass on windows and other
1309 systems where os.sep is not '/'.
1311 * When a cleanUp or tearDown exception occurs, it is now accumulated as a new
1312 traceback in the test details, rather than as a separate call to addError /
1313 addException. This makes testtools work better with most TestResult objects
1314 and fixes bug #335816.
1320 Python 3 support, more matchers and better consistency with Python 2.7 --
1321 you'd think that would be enough for a point release. Well, we here on the
1322 testtools project think that you deserve more.
1324 We've added a hook so that user code can be called just-in-time whenever there
1325 is an exception, and we've also factored out the "run" logic of test cases so
1326 that new outcomes can be added without fiddling with the actual flow of logic.
1328 It might sound like small potatoes, but it's changes like these that will
1329 bring about the end of test frameworks.
1335 * A failure in setUp and tearDown now report as failures not as errors.
1337 * Cleanups now run after tearDown to be consistent with Python 2.7's cleanup
1340 * ExtendedToOriginalDecorator now passes unrecognised attributes through
1341 to the decorated result object, permitting other extensions to the
1342 TestCase -> TestResult protocol to work.
1344 * It is now possible to trigger code just-in-time after an exception causes
1345 a test outcome such as failure or skip. See the testtools MANUAL or
1346 ``pydoc testtools.TestCase.addOnException``. (bug #469092)
1348 * New matcher Equals which performs a simple equality test.
1350 * New matcher MatchesAny which looks for a match of any of its arguments.
1352 * TestCase no longer breaks if a TestSkipped exception is raised with no
1355 * TestCase.run now clones test cases before they are run and runs the clone.
1356 This reduces memory footprint in large test runs - state accumulated on
1357 test objects during their setup and execution gets freed when test case
1358 has finished running unless the TestResult object keeps a reference.
1359 NOTE: As test cloning uses deepcopy, this can potentially interfere if
1360 a test suite has shared state (such as the testscenarios or testresources
1361 projects use). Use the __deepcopy__ hook to control the copying of such
1362 objects so that the shared references stay shared.
1364 * Testtools now accepts contributions without copyright assignment under some
1365 circumstances. See HACKING for details.
1367 * Testtools now provides a convenient way to run a test suite using the
1368 testtools result object: python -m testtools.run testspec [testspec...].
1370 * Testtools now works on Python 3, thanks to Benjamin Peterson.
1372 * Test execution now uses a separate class, testtools.RunTest to run single
1373 tests. This can be customised and extended in a more consistent fashion than
1374 the previous run method idiom. See pydoc for more information.
1376 * The test doubles that testtools itself uses are now available as part of
1377 the testtools API in testtols.testresult.doubles.
1379 * TracebackContent now sets utf8 as the charset encoding, rather than not
1380 setting one and encoding with the default encoder.
1382 * With python2.7 testtools.TestSkipped will be the unittest.case.SkipTest
1383 exception class making skips compatible with code that manually raises the
1384 standard library exception. (bug #490109)
1389 * TestCase.getUniqueInteger is now implemented using itertools.count. Thanks
1390 to Benjamin Peterson for the patch. (bug #490111)
1396 The new matcher API introduced in 0.9.0 had a small flaw where the matchee
1397 would be evaluated twice to get a description of the mismatch. This could lead
1398 to bugs if the act of matching caused side effects to occur in the matchee.
1399 Since having such side effects isn't desirable, we have changed the API now
1400 before it has become widespread.
1405 * Matcher API changed to avoid evaluating matchee twice. Please consult
1406 the API documentation.
1408 * TestCase.getUniqueString now uses the test id, not the test method name,
1409 which works nicer with parameterised tests.
1414 * Python2.4 is now supported again.
1420 This release of testtools is perhaps the most interesting and exciting one
1421 it's ever had. We've continued in bringing together the best practices of unit
1422 testing from across a raft of different Python projects, but we've also
1423 extended our mission to incorporating unit testing concepts from other
1424 languages and from our own research, led by Robert Collins.
1426 We now support skipping and expected failures. We'll make sure that you
1427 up-call setUp and tearDown, avoiding unexpected testing weirdnesses. We're
1428 now compatible with Python 2.5, 2.6 and 2.7 unittest library.
1430 All in all, if you are serious about unit testing and want to get the best
1431 thinking from the whole Python community, you should get this release.
1436 * A new TestResult API has been added for attaching details to test outcomes.
1437 This API is currently experimental, but is being prepared with the intent
1438 of becoming an upstream Python API. For more details see pydoc
1439 testtools.TestResult and the TestCase addDetail / getDetails methods.
1441 * assertThat has been added to TestCase. This new assertion supports
1442 a hamcrest-inspired matching protocol. See pydoc testtools.Matcher for
1443 details about writing matchers, and testtools.matchers for the included
1444 matchers. See http://code.google.com/p/hamcrest/.
1446 * Compatible with Python 2.6 and Python 2.7
1448 * Failing to upcall in setUp or tearDown will now cause a test failure.
1449 While the base methods do nothing, failing to upcall is usually a problem
1450 in deeper hierarchies, and checking that the root method is called is a
1451 simple way to catch this common bug.
1453 * New TestResult decorator ExtendedToOriginalDecorator which handles
1454 downgrading extended API calls like addSkip to older result objects that
1455 do not support them. This is used internally to make testtools simpler but
1456 can also be used to simplify other code built on or for use with testtools.
1458 * New TextTestResult supporting the extended APIs that testtools provides.
1460 * Nose will no longer find 'runTest' tests in classes derived from
1461 testtools.testcase.TestCase (bug #312257).
1463 * Supports the Python 2.7/3.1 addUnexpectedSuccess and addExpectedFailure
1464 TestResult methods, with a support function 'knownFailure' to let tests
1465 trigger these outcomes.
1467 * When using the skip feature with TestResult objects that do not support it
1468 a test success will now be reported. Previously an error was reported but
1469 production experience has shown that this is too disruptive for projects that
1470 are using skips: they cannot get a clean run on down-level result objects.
1473 .. _testtools: http://pypi.python.org/pypi/testtools
git.samba.org / third_party / testtools / blob