Relaxed test discovery for pytest
BSD-2-CLAUSE License
|version| |python| |license| |ci| |coverage|
.. |version| image:: https://img.shields.io/pypi/v/pytest-relaxed :target: https://pypi.org/project/pytest-relaxed/ :alt: PyPI - Package Version .. |python| image:: https://img.shields.io/pypi/pyversions/pytest-relaxed :target: https://pypi.org/project/pytest-relaxed/ :alt: PyPI - Python Version .. |license| image:: https://img.shields.io/pypi/l/pytest-relaxed :target: https://github.com/bitprophet/pytest-relaxed/blob/main/LICENSE :alt: PyPI - License .. |ci| image:: https://img.shields.io/circleci/build/github/bitprophet/pytest-relaxed/main :target: https://app.circleci.com/pipelines/github/bitprophet/pytest-relaxed :alt: CircleCI .. |coverage| image:: https://img.shields.io/codecov/c/gh/bitprophet/pytest-relaxed :target: https://app.codecov.io/gh/bitprophet/pytest-relaxed :alt: Codecov
pytest-relaxed
provides 'relaxed' test discovery for pytest.
It is the spiritual successor to https://pypi.python.org/pypi/spec, but is
built for pytest
instead of nosetests
, and rethinks some aspects of
the design (such as increased ability to opt-in to various behaviors.)
For a development roadmap, see the maintainer's roadmap page <http://bitprophet.org/projects#roadmap>
_.
Has it ever felt strange to you that we put our tests in tests/
, then name
the files test_foo.py
, name the test classes TestFoo
, and finally
name the test methods test_foo_bar
? Especially when almost all of the code
inside of tests/
is, well, tests?
This pytest plugin takes a page from the rest of Python, where you don't have to explicitly note public module/class members, but only need to hint as to which ones are private. By default, all files and objects pytest is told to scan will be considered tests; to mark something as not-a-test, simply prefix it with an underscore.
The "it's a test by default unless underscored" approach works for files::
tests
├── _util.py
├── one_module.py
└── another_module.py
It's applied to module members::
def _helper():
pass
def one_thing():
assert True
def another_thing():
assert False
def yet_another():
assert _helper() == 'something'
And to class members::
class SomeObject:
def behavior_one(self):
assert True
def another_behavior(self):
assert False
def _helper(self):
pass
def it_does_things(self):
assert self._helper() == 'whatever'
As you might expect, there are a few more special cases around discovery to avoid fouling up common test extensions:
conftest.py
aren't treated as tests, because they do specialsetup_(module|class|method|function)
are@pytest.fixture
are, of course,If you like the idea of pytest-relaxed but have a large test suite, it may be daunting to think about "upgrading" it all in one go. It's relatively simple to arrive at a 'hybrid' test suite where your legacy tests still run normally (as long as they're already pytest-compatible, which is true for most unittest suites) but 'relaxed' style tests also work as expected.
The only change you'll still have to make is renaming 'helper' files (any
whose name doesn't start with test_
) so their names begin with an
underscore; then, of course, search and replace any imports of such files.
pytest-relaxed
explicitly sidesteps around anything that looks like
"classic" test files (i.e. named test_*
), allowing pytest's native
collection to take effect. Such files should not need any alteration.
Our reporter (display) functionality still works pretty well with legacy
style tests; test prefixes and suffixes are stripped at display time, so
TestMyThing.test_something
still shows up as if it was written in relaxed
style: MyThing
w/ nested something
.
test_
, and changed any classes to not inherit fromunittest.TestCase
or similar.On top of the relaxed discovery algorithm, pytest-relaxed
also lets you
organize tests in a nested fashion, again like the spec
nose plugin or the
tools that inspired it, such as Ruby's rspec
.
This is purely optional, but we find it's a nice middle ground between having a proliferation of files or suffering a large, flat test namespace making it hard to see which feature areas have been impacted by a bug (or whatnot).
The feature is enabled by using nested/inner classes, like so::
class SomeObject:
def basic_behavior(self):
assert True
class init:
"__init__"
def no_args_required(self):
assert True
def accepts_some_arg(self):
assert True
def sets_up_config(self):
assert False
class some_method:
def accepts_whatever_params(self):
assert False
def base_behavior(self):
assert True
class when_config_says_foo:
def it_behaves_like_this(self):
assert False
class when_config_says_bar:
def it_behaves_like_this(self):
assert True
Test discovery on these inner classes is recursive, so you can nest them as deeply as you like. Naturally, as with all Python code, sometimes you can have too much of a good thing...but that's up to you.
If you're namespacing your tests via nested classes, you may find yourself wanting to reference the enclosing "scope" of the outer classes they live in, such as class attributes. pytest-relaxed automatically copies such attributes onto inner classes during the test collection phase, allowing you to write code like this::
class Outer:
behavior_one = True
def outer_test(self):
assert self.behavior_one
class Inner:
behavior_two = True
def inner_test(self):
assert self.behavior_one and self.behavior_two
Notably:
pytest-relaxed
offers a few other random lightweight test-related utilities
that don't merit their own PyPI entries (most ported from spec
), such as:
trap
, a decorator for use on test functions and/or test
helpers/subroutines which is similar to pytest's own capsys
/capfd
fixtures in that it allows capture of stdout/err.
sys.(stdout|stderr)
withIO
objects which can be getvalue()
'd as needed.raises
, a wrapper around pytest.raises
which works as a decorator,
similar to the Nose testing tool of the same name.
Continuing in the "port of spec
/ inspired by RSpec and friends" vein,
pytest-relaxed
greatly enhances pytest's verbose display mode:
Unlike spec
, this functionality doesn't affect normal/non-verbose output
at all, and can be disabled entirely, allowing you to use the relaxed test
discovery alongside normal pytest verbose display or your favorite pytest
output plugins (such as pytest-sugar
.)
As with most pytest plugins, it's quite simple:
pip install pytest-relaxed
;
Tell pytest where your tests live via the testpaths
option; otherwise
pytest-relaxed will cause pytest to load all of your non-test code as tests!
Not required, but strongly recommended: configure pytest's default
filename pattern (python_files
) to be an unqualified glob (*
).
assert var == othervar
intoassert 17 == 2
during error display) reuses this setting whenThus, a recommended setup.cfg
(or pytest.ini
, sans the header) is::
[tool:pytest] testpaths = tests python_files = *
Write some tests, as exampled above;
pytest
to run the tests, and you're done!