TDD: Test Driven Development
BDD: Behavior Driven Development
smoke test: show that different components of an application are working together
unit test: test most basic feature
integration test: test a larger bit of code (ex: several classes)
system test: check the all system. Something as close to the end-user environment as possible.
functional test: check a single bit of functionality
How to structure the test folder
This one should be roughly the structure of the main code such as shown here.
to run PyTest, simply type py.test (or pytest) at the top level of your project
PyTest will run all the files with names ending in _test.py.
In order to only see the various tests (failing, skipped…etc), use the flag -r as shown here
$ py.test -r f
- f: failed – test did not run successfully
- E: error – exception happened outside of the test function
- s: skipped – test was skipped (see decorators @pytest.mark.skip() and @pytest.mark.skipif())
- x: failed – test was not supposed to pass, ran and failed (good) (see decorators @pytest.mark.xfail())
- X: passed – test was not supposed to pass, ran and passed (not good)
$ py.test -v
With verbose, each row has 1 test
Without verbose, each row has 1 file
$ pytest -k "test_yes_is"
This will look for a test with the test_yes_is string and then run it
$ pip install pytest-cov $ py.test --cov. app/ test/
app/ is the application folder
test/ is the test folder
It’s possible to customized the coverage, for example listing files that do not need to be covered or checked for coverage
[run] omit = *__init__.py
3 configuration files are possible
PyTest comes with a lot of options ($ pytest –help)
|-k Expression||Only run tests/classes which match substring expression|
|-m MARKEXPR||run tests matching mark expression|
|–lf, –last-failed||rerun only the tests that failed last time|
|–ff, –failed-first||run all but rerun failed first|
|-q, –quiet||decrease verbosity|
|–collect-only||collects tests but do not run them|
|–maxfail=num||stop after num failures|
|-x, –exitfirst||stop after first test failing|
$ pytest -k "asdict or defaults" --collect-only ================ test session starts ================== collected 6 items <Module 'tasks/test_four.py'> <Function 'test_asdict'> <Module 'tasks/test_three.py'> <Function 'test_defaults'>
import pytest ... @pytest.mark_run_these_please def test_member_access(): ...
$ pytest -v -m run_these_please ================ test session starts ================== collected 4 items test_four.py::test_replace PASSED test_three.py::test_member_access PASSED ============= 2 tests deselected ======================= ======= 2 passed, 2 deselected in 0.02 seconds ==========